ETL: Neo4jCsvRelationships

Description

Loads data from CSV into Neo4j. Data from CSV are used to insert or update relationships. Driver creates "LOAD CSV" Cypher in a background to load data.

This driver is useful in scenario where there are many columns in CSV and all columns need to be loaded into Neo4j relationships and relationship attribute name is equal to CSV column name. This is also useful when columns in CSV are changed from time to time because this driver loads all columns automatically.

Driver can be configured to insert all data from CSV or update data.

Connection

Attributes

Name

Description

Required

url

url to Neo4j server, Bolt or REST url

yes

user

username for authorization at Neo4j

no

password

password for authorization at Neo4j

no

Parameters

Name

Description

Required

Default

sourceNodeLabels

list of labels to match source node (separated by comma)

yes

targetNodeLabels

list of labels to match target node (separated by comma)

yes

relationshipType

type of relationship to create or update

yes

sourceNodeMatchColumns

pair - name of attribute in Neo4j : name of column in CSV. Which CSV column is used to match Neo4j attribute to match source node

no

targetNodeMatchColumns

pair - name of attribute in Neo4j : name of column in CSV. Which CSV column is used to match Neo4j attribute to match target node

no

sourceNodeMatchDbIdColumn

name of column in CSV which contains database id to match source node

no

targetNodeMatchDbIdColumn

name of column in CSV which contains database id to match target node

no

relationshipMergeColumns

name of columns which are used to merge relationship (separated by comma)

no

relationshipMergeDbIdColumn

name of column which contains database id. This column is used to match relationship by database id.

no

timestampProperties

names of attributes which will be set to current timestamp (separated by comma)

no

emptyStringIsNull

true / false. If true and CSV column contains empty string then null is set into Neo4j attribute. If false and CSV column contains empty string then empty string is set into Neo4j attribute.

no

false

commitSize

size of commit. If it is used then "USING PERIODIC COMMIT" is added to Cypher and data are committed on the fly. If it is not used data are committed at the end.

no

csvPath

path to CSV file with relationships

yes

csvDelimiter

delimiter of columns in CSV file

no

;

Query

Not used.

Script

Load all data from CSV and create or update relationships in Neo4j.

Create

If driver is configured to create data then driver takes all columns from CSV and create new relationships in Neo4j with attributes equal to columns.

Merge

If driver is configured to merge data with Neo4j relationships then driver takes data from CSV and create merge statement with merge columns which are configured.

Data are merged in this way:

  • if there is a relationship in CSV which matches relationship in Neo4j:

    • if CSV contains column and Neo4j contains same attribute then update this attribute

    • if CSV contains column and Neo4j doesn't contain attribute with this name then create this attribute

    • if CSV doesn't contain column with name of some Neo4j attribute then attribute in Neo4j is not changed

  • if there is a relationship in CSV which is not matched in Neo4j: create new relationship in Neo4j

  • if there isn't a relationship in CSV which is in Neo4j: relationship in Neo4j is not changed

Examples

Sample CSV data with relationships:

sourceLogicalName;targetLogicalName;relationshipName;type;subtype
App1;Server1;App1-Server1;Physical;Uses
App2;Server2;App2-Server2;Physical;Uses

Create data (no merge). Match source and target node on columnName. Use column sourceLogicalName to match with attribute logicalName for source node. Use column targetLogicalName to match with attribute logicalName for target node.

<!DOCTYPE etl SYSTEM "http://scriptella.org/dtd/etl.dtd">
<etl>
<description>Relationships from CSV</description>
 
<connection id="relationshipImport" driver="neo4jCsvRelationships" url="bolt://localhost:7687" user="neo4j" password="admin">
sourceNodeLabels=Ci
targetNodeLabels=Ci
relationshipType=RELATED
sourceNodeMatchColumns=logicalName:sourceLogicalName
targetNodeMatchColumns=logicalName:targetLogicalName
timestampProperties=_created,_updated
csvPath=/rels.csv
</connection>
 
<script connection-id="relationshipImport"/>
</etl>

Merge relationships based on column name. Match source and target node on columnName. Use column sourceLogicalName to match with attribute logicalName for source node. Use column targetLogicalName to match with attribute logicalName for target node.

<!DOCTYPE etl SYSTEM "http://scriptella.org/dtd/etl.dtd">
<etl>
<description>Relationships from CSV</description>
 
<connection id="relationshipImport" driver="neo4jCsvRelationships" url="bolt://localhost:7687" user="neo4j" password="admin">
sourceNodeLabels=Ci
targetNodeLabels=Ci
relationshipType=RELATED
sourceNodeMatchColumns=logicalName:sourceLogicalName
targetNodeMatchColumns=logicalName:targetLogicalName
relationshipMergeColumns=relationshipName
timestampProperties=_created,_updated
csvPath=/rels.csv
</connection>
 
<script connection-id="relationshipImport"/>
</etl>