ETL: Neo4j

Description

Executes Cypher at Neo4j Server and return result if required (in query).

Connection

Attributes

Name

Description

Required

url

url to Neo4j server, Bolt or REST url

yes

user

username for authorization at Neo4j

no

password

password for authorization at Neo4j

no

Parameters

Name

Description

Required

Default

query_empty_value

string that represents what will be returned if null value of attribute is returned from Neo4j

no

If null for attribute is returned from Neo4j then use name of this attribute

Query

Executes Cypher and return result. Use for read Cypher statements.

You can use any script in Neo4j query element to process every record returned by query. Attributes from query result are accessible as "$name_of_attribute".

Script

Executes Cypher. Use for modification Cypher statements. All statements of one connector are executed in the same Neo4j transaction and this one transaction is committed at the end of ETL job. If there is some error at execution of ETL script then rollback is called for transaction at Neo4j and ETL is ended. Rollback is not working if you use Cypher "USING PERIODIC COMMIT".

If you need more transactions create new connector in the same ETL or use attribute new-tx="true" in script element in the same connector. Attribute "new-tx" will cause that new separate transaction will be created and committed. This transaction is independent of main script transaction.

USING PERIODIC COMMIT

If you need to use "USING PERIODIC COMMIT" Cypher statement this statement has to be only one statement executed in script. Use "new-tx" attribute or more connectors to execute more Cypher queries in the same ETL.

Examples

Query example: Executes Cypher query and every record from result write into text file. Script element is used in Query element to process every record of result.

<!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
<etl>
<description>Test neo4j query</description>
<connection id="neo4j" driver="neo4j" url="bolt://localhost:7687" user="neo4j" password="admin">
query_empty_value=
</connection>
<connection id="out" driver="text" url="d:\\testfile.txt"/>
<query connection-id="neo4j">
MATCH (n:Ci) RETURN id(n) as id, n.logicalName as logicalName LIMIT 25
<script connection-id="out">$rownum;$id;$logicalName</script>
</query>
</etl>

Script example: Executes Cypher statement.

<!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
<etl>
<description>Test neo4j query</description>
<connection id="neo4j" driver="neo4j" url="bolt://localhost:7687" user="neo4j" password="admin">
</connection>
<script connection-id="neo4j">
CREATE INDEX ON :Ci(logicalName)
</script>
</etl>

Script example: Executes Cypher statement based on result of SQL Query. Script element is used in Query element to process every record of result. Neo4j transaction is committed at the end of ETL script.

Every record of SQL result is inserted into Neo4j directly. This is not best for performance and this example only show possibilities of Neo4j driver. For better performance write result of SQL into CSV file and then use "LOAD CSV" statement instead (see ETL job examples).

<!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
<etl>
<description>DB to Neo4j</description>
<properties>
db.driver=postgresql
db.url=jdbc:postgresql://localhost:5432/cmdb
db.user=someuser
db.password=somepassword
</properties>
<connection id="db" driver="$db.driver" url="$db.url" user="$db.user" password="$db.password" />
<connection id="neo4j" driver="neo4j" url="bolt://localhost:7687" user="neo4j" password="admin"/>
<query connection-id="db">
select
case when id_sm is not null then id_sm else '' end as id_sm,
logical_name,
title,
case when type is not null then type else '' end as type,
case when subtype is not null then subtype else '' end as subtype
from device2m1
<script connection-id="neo4j">
CREATE (n:Ci) SET n.idSm = '$id_sm', n.logicalName = '$logical_name', n.type = '$type', n.subtype = '$subtype'
</script>
</query>
</etl>