Let's migrate graph to Neo4j Aura
Dou Liu
 2021-11-04 18:45:41  712 Words  4 Mins

During the user evaluation of conversational virtual museum guide, one of the user pointed that the knowledge graph visualization was not working. The visualization of knowledge graph is meant to show the connection between entities in knowledge graph directly, to help user understand their relationships.

The Problem is …

In my project, the visualization is implemented using Neovis.js package. It allows the graph visualization in the browser with data from Neo4j. In the config, it requires to connect with a Neo4j database, and in my case, it was connected with my local knowledge graph database, something like this:


var config = {
                    container_id: "viz",
                    server_url: "bolt://localhost:6784",
                    server_user: "neo4j",
                    server_password: "password",
                    ...
                    }

Therefore, when the user opens this page to view the visualization, the plug-in will perform queries on the local graph database based on the configuration above. The problem is that there is no graph database on the user’s computer. So, here I need to configure a database that can be accessed remotely.

Possible way to solve it

As long as we have a remote database, or make my local database accessible remotely, this problem will be solved. There are two possible methods for it, as far as I concerned:

  1. Export my local knowledge base and upload it to a remote server
  2. Use tools like Ngrok to make localhost accessible by Internet

Finally I chose the first method. Since the second way has an obvious shortcoming that my pc would not be a good server because it is not guaranteed to be accessible 7/24.

And to find a remote server, it could be cloud server from Google or Amazon. But the complex registration and configuration would be too much work for a small project, unless you already have your own server, and the price… I don’t know.

Fortunately, Neo4j has it own cloud service, which is called Neo4j Aura, and Aura has free plan for personal use!

Hands on

First, we need to export the knowledge graph data from local database.

Export the data

I tried apoc.export.all.csv first, however, it didn’t work really well. The exported csv data can not be read properly on Aura.

Another way is generating a dump file. This way is much easier and promising. Open the Terminal of your graph database on Neo4j Desktop. And the command to generate dump is:

$ bin\neo4j-admin dump --database=neo4j --to={target path }.dump

Result:

Done: 89 files, 63.31MiB processed.

Import the data on Aura

Before importing, create a project on Aura, and click the title to see the import section:

Aura upload

Here, drop the Dump file to upload, wait for a while for the updating and then the knowledge graph is on the cloud server now.

The URL for this database is offered. With the URL, name and password, now the knowledge graph can be accessed anywhere!

Issue for the URL

The default URL of the database on Aura starts with neo4j+s:// , when I use this as the configuration in Neovis, it specifies to require SSL and a valid cert. As suggested in the document of Neovis

config.encrypted

"ENCRYPTION_OFF"` (default) or `"ENCRYPTION_ON"

This must be set to "ENCRYPTION_ON" when using a secure WebSocket connection, such as with Neo4j Aura.

I added it in the config, and the page got error said:

Encryption/trust can only be configured either through URL or config

To solve it, as suggested in here, set the URL to be neo4j:// fixes it. Like this:

var config = {
                    container_id: "viz",
                    server_url: "neo4j://fc932d9f.databases.neo4j.io",
                    server_user: "neo4j",
                    server_password: "password",
                    encrypted:'ENCRYPTION_ON',
                    trust:'TRUST_SYSTEM_CA_SIGNED_CERTIFICATES',
					...
				}

Connect Aura with Python

So as I transfer the dataset to Aura, I should also update my python code, which is connected to the graph base for query using Py2neo.

The previous code to connect local database is:

g = Graph(
            scheme="neo4j",
            host="localhost",
            port=7687,
            auth=("neo4j", "password"))

For Aura, it changes to:

g = Graph("neo4j+s://fc932d9f.databases.neo4j.io", auth=("neo4j", "password"), routing=True)
 Comments