ETL: Neo4j
Description
Executes Cypher at Neo4j Server and return result if required (in query).
Connection
Attributes
Name |
Description |
Required |
url |
url to Neo4j server, Bolt or REST url |
yes |
user |
username for authorization at Neo4j |
no |
password |
password for authorization at Neo4j |
no |
Parameters
Name |
Description |
Required |
Default |
query_empty_value |
string that represents what will be returned if null value of attribute is returned from Neo4j |
no |
If null for attribute is returned from Neo4j then use name of this attribute |
Query
Executes Cypher and return result. Use for read Cypher statements.
You can use any script in Neo4j query element to process every record returned by query. Attributes from query result are accessible as "$name_of_attribute".
Script
Executes Cypher. Use for modification Cypher statements. All statements of one connector are executed in the same Neo4j transaction and this one transaction is committed at the end of ETL job. If there is some error at execution of ETL script then rollback is called for transaction at Neo4j and ETL is ended. Rollback is not working if you use Cypher "USING PERIODIC COMMIT".
If you need more transactions create new connector in the same ETL or use attribute new-tx="true" in script element in the same connector. Attribute "new-tx" will cause that new separate transaction will be created and committed. This transaction is independent of main script transaction.
USING PERIODIC COMMIT
If you need to use "USING PERIODIC COMMIT" Cypher statement this statement has to be only one statement executed in script. Use "new-tx" attribute or more connectors to execute more Cypher queries in the same ETL.
Examples
Query example: Executes Cypher query and every record from result write into text file. Script element is used in Query element to process every record of result.
<!DOCTYPE etl SYSTEM "
https://scriptella.org/dtd/etl.dtd
">
<
etl
>
<
description
>Test neo4j query</
description
>
<
connection
id
=
"neo4j"
driver
=
"neo4j"
url
=
"bolt://localhost:7687"
user
=
"neo4j"
password
=
"admin"
>
query_empty_value=
</
connection
>
<
connection
id
=
"out"
driver
=
"text"
url
=
"d:\\testfile.txt"
/>
<
query
connection-id
=
"neo4j"
>
MATCH (n:Ci) RETURN id(n) as id, n.logicalName as logicalName LIMIT 25
<
script
connection-id
=
"out"
>$rownum;$id;$logicalName</
script
>
</
query
>
</
etl
>
Script example: Executes Cypher statement.
<!DOCTYPE etl SYSTEM "
https://scriptella.org/dtd/etl.dtd
">
<
etl
>
<
description
>Test neo4j query</
description
>
<
connection
id
=
"neo4j"
driver
=
"neo4j"
url
=
"bolt://localhost:7687"
user
=
"neo4j"
password
=
"admin"
>
</
connection
>
<
script
connection-id
=
"neo4j"
>
CREATE INDEX ON :Ci(logicalName)
</
script
>
</
etl
>
Script example: Executes Cypher statement based on result of SQL Query. Script element is used in Query element to process every record of result. Neo4j transaction is committed at the end of ETL script.
Every record of SQL result is inserted into Neo4j directly. This is not best for performance and this example only show possibilities of Neo4j driver. For better performance write result of SQL into CSV file and then use "LOAD CSV" statement instead (see ETL job examples).
<!DOCTYPE etl SYSTEM "
https://scriptella.org/dtd/etl.dtd
">
<
etl
>
<
description
>DB to Neo4j</
description
>
<
properties
>
db.driver=postgresql
db.url=jdbc:
postgresql://localhost:5432/cmdb
db.user=someuser
db.password=somepassword
</
properties
>
<
connection
id
=
"db"
driver
=
"$db.driver"
url
=
"$db.url"
user
=
"$db.user"
password
=
"$db.password"
/>
<
connection
id
=
"neo4j"
driver
=
"neo4j"
url
=
"bolt://localhost:7687"
user
=
"neo4j"
password
=
"admin"
/>
<
query
connection-id
=
"db"
>
select
case when id_sm is not null then id_sm else '' end as id_sm,
logical_name,
title,
case when type is not null then type else '' end as type,
case when subtype is not null then subtype else '' end as subtype
from device2m1
<
script
connection-id
=
"neo4j"
>
CREATE (n:Ci) SET n.idSm = '$id_sm', n.logicalName = '$logical_name', n.type = '$type', n.subtype = '$subtype'
</
script
>
</
query
>
</
etl
>