EvaluateJsonPath

Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 3206A200D14 for ; Tue, 3 Oct 2017 15:30:48 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 30C021609DD; Tue, 3 Oct 2017 13:30:48 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 87092160BE1 for ; Tue, 3 Oct 2017 15:30:45 +0200 (CEST) Received: (qmail 31724 invoked by uid 500); 3 Oct 2017 13:30:39 -0000 Mailing-List: contact commits-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@nifi.apache.org Delivered-To: mailing list commits@nifi.apache.org Received: (qmail 31522 invoked by uid 99); 3 Oct 2017 13:30:39 -0000 Received: from Unknown (HELO svn01-us-west.apache.org) (209.188.14.144) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Oct 2017 13:30:39 +0000 Received: from svn01-us-west.apache.org (localhost [127.0.0.1]) by svn01-us-west.apache.org (ASF Mail Server at svn01-us-west.apache.org) with ESMTP id D93FA3A1382 for ; Tue, 3 Oct 2017 13:30:34 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1811008 [20/43] - in /nifi/site/trunk/docs: ./ nifi-docs/ nifi-docs/components/ nifi-docs/components/org.apache.nifi/ nifi-docs/components/org.apache.nifi/nifi-ambari-nar/ nifi-docs/components/org.apache.nifi/nifi-ambari-nar/1.4.0/ nifi-do... Date: Tue, 03 Oct 2017 13:30:27 -0000 To: commits@nifi.apache.org From: jstorck@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20171003133034.D93FA3A1382@svn01-us-west.apache.org> archived-at: Tue, 03 Oct 2017 13:30:48 -0000 Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.EvaluateJsonPath/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.EvaluateJsonPath/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.EvaluateJsonPath/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.EvaluateJsonPath/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +EvaluateJsonPath

Description:

Evaluates one or more JsonPath expressions against the content of a FlowFile. The results of those expressions are assigned to FlowFile Attributes or are written to the content of the FlowFile itself, depending on configuration of the Processor. JsonPaths are entered by adding user-defined properties; the name of the property maps to the Attribute Name into which the result will be placed (if the Destination is flowfile-attribute; otherwise, the property name is ignored). The value of the property must be a valid JsonPath expression. A Return Type o f 'auto-detect' will make a determination based off the configured destination. When 'Destination' is set to 'flowfile-attribute,' a return type of 'scalar' will be used. When 'Destination' is set to 'flowfile-content,' a return type of 'JSON' will be used.If the JsonPath evaluates to a JSON array or JSON object and the Return Type is set to 'scalar' the FlowFile will be unmodified and will be routed to failure. A Return Type of JSON can return scalar values if the provided JsonPath evaluates to the specified value and will be routed as a match.If Destination is 'flowfile-content' and the JsonPath does not evaluate to a defined path, the FlowFile will be routed to 'unmatched' without having its contents modified. If Destination is flowfile-attribute and the expression matches nothing, attributes will be created with empty strings as the value, and the FlowFile will always be routed to 'matched.'

Additional Details...

Tags:

JS ON, evaluate, JsonPath

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values.

Name	Default Value	Allowable Values	Description
Destination	flowfile-content	flowfile-content flowfile-attribute	Indicates whether the results of the JsonPath evaluation are written to the FlowFile content or a FlowFile attribute; if using attribute, must specify the Attribute Name property. If set to flowfile-content, only one JsonPath may be specified, and the property name is ignored.
Return Type	auto-detect	auto-detect jso n scalar	Indicates the desired return type of the JSON Path expressions. Selecting 'auto-detect' will set the return type to 'json' for a Destination of 'flowfile-content', and 'scalar' for a Destination of 'flowfile-attribute'.
Path Not Found Behavior	ignore	warn ignore	Indicates how to handle missing JSON path expressions when destination is set to 'flowfile-attribute'. Selecting 'warn' will generate a warning when a JSON path expression is not found.
Null Value Representation	empty string	empty string the string 'null'	Indicates the desired representation of JSON Path expressions resulting in a null value.

Dynamic Pro perties:

Dynamic Properties allow the user to specify both the name and value of a property.
Name Value Description
A FlowFile attribute(if <Destination> is set to 'flowfile-attribute') A JsonPath expression If <Destination>='flowfile-attribute' then that FlowFile attribute will be set to any JSON objects that match the JsonPath. If <Destination>='flowfile-content' then the FlowFile content will be updated to any JSON objects that match the JsonPath.

Name	Value	Description
A FlowFile attribute(if <Destination> is set to 'flowfile-attribute')	A JsonPath expression	If <Destination>='flowfile-attribute' then that FlowFile attribute will be set to any JSON objects that match the JsonPath. If <Destination>='flowfile-content' then the FlowFile content will be updated to any JSON objects that match the JsonPath.

Relationships:

Name	Description
failure	FlowFiles are routed to this relationship when the JsonPath cannot be evaluated against the content of the FlowFile; for instance, if the FlowFile is not valid JSON
unmatched	FlowFiles are routed to this relationship when the JsonPath does not m atch the content of the FlowFile and the Destination is set to flowfile-content
matched	FlowFiles are routed to this relationship when the JsonPath is successfully evaluated and the FlowFile is modified as a result

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.EvaluateXPath/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.EvaluateXPath/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.EvaluateXPath/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.EvaluateXPath/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +EvaluateXPath

Description:

Evaluates one or more XPaths against the content of a FlowFile. The results of those XPaths are assigned to FlowFile Attributes or are written to the content of the FlowFile itself, depending on configuration of the Processor. XPaths are entered by adding user-defined properties; the name of the property maps to the Attribute Name into which the result will be placed (if the Destination is flowfile-attribute; otherwise, the property name is ignored). The value of the property must be a valid XPath expression. If the XPath evaluates to more than one node a nd the Return Type is set to 'nodeset' (either directly, or via 'auto-detect' with a Destination of 'flowfile-content'), the FlowFile will be unmodified and will be routed to failure. If the XPath does not evaluate to a Node, the FlowFile will be routed to 'unmatched' without having its contents modified. If Destination is flowfile-attribute and the expression matches nothing, attributes will be created with empty strings as the value, and the FlowFile will always be routed to 'matched'

Tags:

XML, evaluate, XPath

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values.

Name	Default Value	Allowable Values	Description
Destination	flowfile-content	fl owfile-content flowfile-attribute	Indicates whether the results of the XPath evaluation are written to the FlowFile content or a FlowFile attribute; if using attribute, must specify the Attribute Name property. If set to flowfile-content, only one XPath may be specified, and the property name is ignored.
Return Type	auto-detect	auto-detect nodeset string	Indicates the desired return type of the Xpath expressions. Selecting 'auto-detect' will set the return type to 'nodeset' for a Destination of 'flowfile-content', and 'string' for a Destination of 'flowfile-attribute'.
Validate DTD	true	true false	Specifies whether or not the XML con tent should be validated against the DTD.

Dynamic Properties:

Dynamic Properties allow the user to specify both the name and value of a property.
Name Value Description
A FlowFile attribute(if <Destination> is set to 'flowfile-attribute' An XPath expression If <Destination>='flowfile-attribute' then the FlowFile attribute is set to the result of the XPath Expression. If <Destination>='flowfile-content' then the FlowFile content is set to the result of the XPath Expression.

Name	Value	Description
A FlowFile attribute(if <Destination> is set to 'flowfile-attribute'	An XPath expression	If <Destination>='flowfile-attribute' then the FlowFile attribute is set to the result of the XPath Expression. If <Destination>='flowfile-content' then the FlowFile content is set to the result of the XPath Expression.

Relationships:

Name	Description
failure	FlowFiles are routed to this relationship when the XPath cannot be evaluated against the content of the FlowFile; for instance, if the FlowFile is not valid XML, or if the Return Type is 'nodeset' and the XPath evaluates to mu ltiple nodes
unmatched	FlowFiles are routed to this relationship when the XPath does not match the content of the FlowFile and the Destination is set to flowfile-content
matched	FlowFiles are routed to this relationship when the XPath is successfully evaluated and the FlowFile is modified as a result

Reads Attributes:

None specified.

Writes Attributes:

Name	Description
user-defined	This processor adds user-defined attributes if the <Destination> property is set to flowfile-attribute.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.EvaluateXQuery/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.EvaluateXQuery/additionalDetails.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.EvaluateXQuery/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.EvaluateXQuery/additionalDetails.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1,149 @@ + + + + + + EvaluateXQuery + + + + + + + + + +

+ Examples: +

+ +

This processor produces one attribute or FlowFile per + XQueryResult. If only one attribute or FlowFile is desired, the + following examples demonstrate how this can be achieved using the + XQuery language. The examples below reference the following sample + XML:

+ +

+		
+  <?xml version="1.0" encoding="UTF-8"?>
+  <?xml-stylesheet type="text/xsl" href="foo.xsl"?>
+  <ns:fruitbasket xmlns:ns="http://namespace/1">
+    <fruit taste="crisp">    
+      <!-- Apples are my favorite-->   
+      <name>apple</name>   
+      <color>red</color>  
+    </fruit>  
+    <fruit>   
+      <name>apple</name>   
+      <color>green</color>  
+    </fruit>  
+    <fruit>   
+      <name>banana</name>   
+      <color>yellow</color>  
+    </fruit>  
+    <fruit taste="sweet">   
+      <name>orange</name>   
+      <color>orange</color>  
+    </fruit>  
+    <fruit>   
+      <name>blueberry</name>   
+      <color>blue</color>  
+    </fruit>  
+      <fruit taste="tart">   
+      <name>raspberry</name>   
+      <color>red</color>  
+    </fruit>  
+    <fruit>   
+      <name>none</name>    
+      <color/>  
+    </fruit>
+  </ns:fruitbasket>
+
+

+ +

XQuery to return all "fruit" nodes individually (7 Results): +
- //fruit
+
XQuery to return only the first "fruit" node (1 Result): +
- //fruit[1]
+
XQuery to return only the last "fruit" node (1 Result): +
- //fruit[count(//fruit)]
+
XQuery to return all "fruit" nodes, wrapped in a "basket" tag + (1 Result): +
- <basket>{//fruit}</basket>
+
XQuery to return all "fruit" names individually (7 Results): +
- //fruit/text()
+
XQuery to return only the first "fruit" name (1 Result): +
- //fruit[1]/text()
+
XQuery to return only the last "fruit" name (1 Result): +
- //fruit[count(//fruit)]/text()
+
XQuery to return all "fruit" names as a comma separated list + (1 Result): +
- string-join((for $x in //fruit return $x/name/text()), ', + ')
+
XQuery to return all "fruit" colors and names as a comma + separated list (1 Result): +
- string-join((for $y in (for $x in //fruit return + string-join(($x/color/text() , $x/name/text()), ' ')) return $y), + ', ')
+
XQuery to return all "fruit" colors and names as a comma + separated list (1 Result): +
- string-join((for $y in (for $x in //fruit return + string-join(($x/color/text() , $x/name/text()), ' ')) return $y), + ', ')
+
XQuery to return all "fruit" colors and names as a new line + separated list (1 Result): +
- string-join((for $y in (for $x in //fruit return + string-join(($x/color/text() , $x/name/text()), ' ')) return $y), + '\n')
+

+ + Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.EvaluateXQuery/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.EvaluateXQuery/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.EvaluateXQuery/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.EvaluateXQuery/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +EvaluateXQuery

Description:

Evaluates one or more XQueries against the content of a FlowFile. The results of those XQueries are assigned to FlowFile Attributes or are written to the content of the FlowFile itself, depending on configuration of the Processor. XQueries are entered by adding user-defined properties; the name of the property maps to the Attribute Name into which the result will be placed (if the Destination is 'flowfile-attribute'; otherwise, the property name is ignored). The value of the property must be a valid XQuery. If the XQuery returns more than one result , new attributes or FlowFiles (for Destinations of 'flowfile-attribute' or 'flowfile-content' respectively) will be created for each result (attributes will have a '.n' one-up number appended to the specified attribute name). If any provided XQuery returns a result, the FlowFile(s) will be routed to 'matched'. If no provided XQuery returns a result, the FlowFile will be routed to 'unmatched'. If the Destination is 'flowfile-attribute' and the XQueries matche nothing, no attributes will be applied to the FlowFile.

Additional Details...

Tags:

XML, evaluate, XPath, XQuery

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values.

Name	Default Value	Allowable Values	Description
Des tination	flowfile-content	flowfile-content flowfile-attribute	Indicates whether the results of the XQuery evaluation are written to the FlowFile content or a FlowFile attribute. If set to <flowfile-content>, only one XQuery may be specified and the property name is ignored. If set to <flowfile-attribute> and the XQuery returns more than one result, multiple attributes will be added to theFlowFile, each named with a '.n' one-up number appended to the specified attribute name
Output: Method	xml	xml html text	Identifies the overall method that should be used for outputting a result tree.
Output: Omit XML Declaration	false		Specifies whether the processor should output an XML declaration when transforming a result tree.
Output: Indent	false		Specifies whether the processor may add additional whitespace when outputting a result tree.
Validate DTD	true	true false	Specifies whether or not the XML content should be validated against the DTD.

Dynamic Properties:

Dynamic Properties allow the user to specify both the name and value of a property.
Name Value Description
A FlowFile attribute(if <Destination> is set to 'flowfile-attribute' An XQuery If <Destination>='flowfile-attribute' then the FlowFile attribute is set to the result of the XQuery. If <Destination>='flowfile-content' then the FlowFile content is set to the result of the XQuery.

Name	Value	Description
A FlowFile attribute(if <Destination> is set to 'flowfile-attribute'	An XQuery	If <Destination>='flowfile-attribute' then the FlowFile attribute is set to the result of the XQuery. If <Destination>='flowfile-content' then the FlowFile content is set to the result of the XQuery.

Relationships:

Name	Description
failure	FlowFiles are routed to this relationship when the XQuery cannot be evaluated against the content of the FlowFile.
unmatched	FlowFiles are routed to this relationship when the XQuery does not match the content of the FlowFile and the Destination is set to flowfile-content
matched	FlowFiles are routed to this relationship when the XQuery is successfully evaluated and the FlowFile is modified as a result

Reads Attributes:

None specified.

Writes Attributes:

Name	Description
user-defined	This p rocessor adds user-defined attributes if the <Destination> property is set to flowfile-attribute .

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExecuteProcess/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExecuteProcess/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExecuteProcess/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExecuteProcess/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +ExecuteProcess

Description:

Runs an operating system command specified by the user and writes the output of that command to a FlowFile. If the command is expected to be long-running, the Processor can output the partial data on a specified interval. When this option is used, the output is expected to be in textual format, as it typically does not make sense to split binary data on arbitrary time-based intervals.

Tags:

command, process, source, external, invoke, script, restricted

Properties:

In the list below, the names of required properties appear i n bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

Name	Default Value	Allowable Values	Description
Command			Specifies the command to be executed; if just the name of an executable is provided, it must be in the user's environment PATH.
Command Arguments			The arguments to supply to the executable delimited by white space. White space can be escaped by enclosing it in double-quotes. Supports Expression Language: true
Batch Duration			If the process is expected to be long-running and produce textual output, a batch duration can be specified so that the output will be captured for this amount of time and a FlowFile will then be sent out with the results and a new FlowFile will be started, rather than waiting for the process to finish before sending out the results
Redirect Error Stream	false	true false	If true will redirect any error stream output of the process to the output stream. This is particularly helpful for processes which write extensively to the error stream or for troubleshooting.
Argument Delimiter			Delimiter to use to separate arguments for a command [default: space]. Must be a single character.

Dynamic Properties:

Dynamic Properties allow the user to specify both the name and value of a property.
Name Value Description
An environment variable name An environment variable value These environment variables are passed to the process spawned by this Processor

Name	Value	Description
An environment variable name	An environment variable value	These environment variables are passed to the process spawned by this Processor

Relationships:

Name	Description
success	All created FlowFiles are routed to this relationship

Reads Attributes:

None specified.

Writes Attributes:

Name	Description
command	Executed command
command.arguments	Arguments of the command

State management:

This component does not store state.

Restricted:

Provides op erator the ability to execute arbitrary code assuming all permissions that NiFi has.

Input requirement:

This component does not allow an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExecuteSQL/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExecuteSQL/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExecuteSQL/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExecuteSQL/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +ExecuteSQL

Description:

Execute provided SQL select query. Query result will be converted to Avro format. Streaming is used so arbitrarily large result sets are supported. This processor can be scheduled to run on a timer, or cron expression, using the standard scheduling methods, or it can be triggered by an incoming FlowFile. If it is triggered by an incoming FlowFile, then attributes of that FlowFile will be available when evaluating the select query. FlowFile attribute 'executesql.row.count' indicates how many rows were selected.

Tags:

sql, select, jdbc, query, data base

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

Name	Default Value	Allowable Values	Description
Database Connection Pooling Service		Controller Service API: DBCPService Implementations: DBCPConnectionPool HiveConnectionPool	The Controller Service that is used to o btain connection to database
SQL select query			The SQL select query to execute. The query can be empty, a constant value, or built from attributes using Expression Language. If this property is specified, it will be used regardless of the content of incoming flowfiles. If this property is empty, the content of the incoming flow file is expected to contain a valid SQL select query, to be issued by the processor to the database. Note that Expression Language is not evaluated for flow file contents. Supports Expression Language: true
Max Wait Time	0 seconds		The maximum amount of time allowed for a running SQL select query , zero means there is no limit. Max time less than 1 second will be equal to zero.
< strong>Normalize Table/Column Names	false	true false	Whether to change non-Avro-compatible characters in column names to Avro-compatible characters. For example, colons and periods will be changed to underscores in order to build a valid Avro record.
Use Avro Logical Types	false	true false	Whether to use Avro Logical Types for DECIMAL/NUMBER, DATE, TIME and TIMESTAMP columns. If disabled, written as string. If enabled, Logical types are used and written as its underlying type, specifically, DECIMAL/NUMBER as logical 'decimal': written as bytes with additional precision and scale meta data, DATE as logical 'date-millis': written as int denoting days since Unix epoch (1970-01-01), TIME as logical 'time-millis': written a s int denoting milliseconds since Unix epoch, and TIMESTAMP as logical 'timestamp-millis': written as long denoting milliseconds since Unix epoch. If a reader of written Avro records also knows these logical types, then these values can be deserialized with more context depending on reader implementation.
Default Decimal Precision	10		When a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'precision' denoting number of available digits is required. Generally, precision is defined by column data type definition or database engines default. However undefined precision (0) can be returned from some database engines. 'Default Decimal Precision' is used when writing those undefined precision numbers. Supports Expression Language: true
Default Decimal Scale	0		When a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'scale' denoting number of available decimal digits is required. Generally, scale is defined by column data type definition or database engines default. However when undefined precision (0) is returned, scale can also be uncertain with some database engines. 'Default Decimal Scale' is used when writing those undefined numbers. If a value has more decimals than specified scale, then the value will be rounded-up, e.g. 1.53 becomes 2 with scale 0, and 1.5 with scale 1. Supports Expression Language: true

Relationships:

Name	Description
success	Successfully created FlowFile from SQL query result set.
failure	SQL query execution failed. Incoming FlowFile will be penalized and routed to this relationsh ip

Reads Attributes:

None specified.

Writes Attributes:

Name	Description
executesql.row.count	Contains the number of rows returned in the select query
executesql.query.duration	Duration of the query in milliseconds

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component allows an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExecuteStreamCommand/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExecuteStreamCommand/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExecuteStreamCommand/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExecuteStreamCommand/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +ExecuteStreamCommand

Description:

Executes an external command on the contents of a flow file, and creates a new flow file with the results of the command.

Tags:

command execution, command, stream, execute, restricted

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

< table id="properties">NameDefault ValueAllowable ValuesDescriptionCommand ArgumentsThe arguments to supply to the executable delimited by the ';' character.
Supports Expression Language: trueCommand PathSpecifies the command to be executed; if just the name of an executable is provided, it must be in the user's environment PATH.
Supports Expression Language: trueIgnore STDINfalse

true
false

If true, the contents of the incoming flowfile will not be passed to the executing commandWorking DirectoryThe directory to use as the current working directory when executing the command
Supports Expression Language: trueArgument Delimiter;Delimiter to use to separate arguments for a command [default: ;]. Must be a single characterOutput Destination AttributeIf set, the output of the stream command will be put into an attribute of the original FlowFile instead of a separate FlowFile. There will no longer be a relationship for 'output stream'. The value of this property will be the key for the output attribute.Max Attribute Length256If routing t he output of the stream command to an attribute, the number of characters put to the attribute value will be at most this amount. This is important because attributes are held in memory and large attributes will quickly cause out of memory issues. If the output goes longer than this value, it will truncated to fit. Consider making this smaller if able.

Dynamic Properties:

Name	Value	Description
An environment variable name	An environment variable value	These environment variables are passed to the process spawned by this Processor

Relationships:

Name	Description
output stream	The destination path for the flow file created from the command's output
origin al	FlowFiles that were successfully processed

Reads Attributes:

None specified.

Writes Attributes:

Name	Description
execution.command	The name of the command executed
execution.command.args	The semi-colon delimited list of arguments
execution.status	The exit status code returned from executing the command
execution.error	Any error messages returned from executing the command

State management:

This component does not store state.

Restricted:

Provides operator the ability to execute arbitrary code assuming all permissions that NiFi has.

Input requirement:

This component requires an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExtractGrok/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExtractGrok/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExtractGrok/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExtractGrok/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +ExtractGrok

Description:

Evaluates one or more Grok Expressions against the content of a FlowFile, adding the results as attributes or replacing the content of the FlowFile with a JSON notation of the matched content

Tags:

grok, log, text, parse, delimit, extract

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values.

Name	Default Value	Allowable Values	Description
Grok Expression			Grok expression
Grok Pattern file			Grok Pattern file definition
Destination	flowfile-attribute	flowfile-attribute flowfile-content	Control if Grok output value is written as a new flowfile attributes, in this case each of the Grok identifier that is matched in the flowfile will be added as an attribute, prefixed with "grok." or written in the flowfile content. Writing to flowfile content will overwrite any existing flowfile content.
Character Set	UTF-8		The Character Set in which the file is encoded
Maximum Buffer Size	1 MB		Specifies the maximum amount of data to buffer (per file) in order to apply the Grok expressions. Files larger than the specified maximum will not be fully evaluated.

Relationships:

Name	Description
unmatched	FlowFiles are routed to this relationship when no provided Grok Expression matches the content of the FlowFile
matched	FlowFiles are routed to this relationship when the Grok Expression is successfully evaluated and the FlowFile is modified as a result

Reads Attributes:

None specified.

Writes Attributes:

Name	Description
grok.XXX	When operati ng in flowfile-attribute mode, each of the Grok identifier that is matched in the flowfile will be added as an attribute, prefixed with "grok." For example,if the grok identifier "timestamp" is matched, then the value will be added to an attribute named "grok.timestamp"

State management:

This component does not store state.

Restricted:

This component is not restricted. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExtractText/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExtractText/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExtractText/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.ExtractText/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1,2 @@ +ExtractText

Description:

Evaluates one or more Regular Expressions against the content of a FlowFile. The results of those Regular Expressions are assigned to FlowFile Attributes. Regular Expressions are entered by adding user-defined properties; the name of the property maps to the Attribute Name into which the result will be placed. The first capture group, if any found, will be placed into that attribute name.But all capture groups, including the matching string sequence itself will also be provided at that attribute name with an index value provided, with the exception of a ca pturing group that is optional and does not match - for example, given the attribute name "regex" and expression "abc(def)?(g)" we would add an attribute "regex.1" with a value of "def" if the "def" matched. If the "def" did not match, no attribute named "regex.1" would be added but an attribute named "regex.2" with a value of "g" will be added regardless.The value of the property must be a valid Regular Expressions with one or more capturing groups. If the Regular Expression matches more than once, only the first match will be used unless the property enabling repeating capture group is set to true. If any provided Regular Expression matches, the FlowFile(s) will be routed to 'matched'. If no provided Regular Expression matches, the FlowFile will be routed to 'unmatched' and no attributes will be applied to the FlowFile.

Tags:

evaluate, extract, Text, Regular Expression, regex

Properties:

In the list below, the names of required properties appear in ~~bold. Any other properties (not in bold) are considered optional. The table also indicates any default values.~~

Name	Default Value	Allowable Values	Description
Character Set	UTF-8		The Character Set in which the file is encoded
Maximum Buffer Size	1 MB		Specifies the maximum amount of data to buffer (per file) in order to apply the regular expressions. Files larger than the specified maximum will not be fully evaluated.
Maximum Capture Group Length	1024		Specifies the maximum number of characters a given capture group value can have. Any characters beyond the max will be truncated.
Enable Canonical Equivalence	false	true false	Indicates that two characters match only when their full canonical decompositions match.
Enable Case-insensitive Matching	false	true false	Indicates that two characters match even if they are in a different case. Can also be specified via the embedded flag (?i).
Permit Whitespace and Comments in Pattern	false	true false	In this mode, whitespace is ignored, and embedded comments starting with # are ignored until the end of a line. Can also be specified via the em bedded flag (?x).
Enable DOTALL Mode	false	true false	Indicates that the expression '.' should match any character, including a line terminator. Can also be specified via the embedded flag (?s).
Enable Literal Parsing of the Pattern	false	true false	Indicates that Metacharacters and escape characters should be given no special meaning.
Enable Multiline Mode	false	true false	Indicates that '^' and '$' should match just after and just before a line terminator or end of sequence, instead of only the beginning or end of the entire input. Can also be specified via the embeded flag (?m).
Enable Unicode-aware Case Folding	false	true false	When used with 'Enable Case-insensitive Matching', matches in a manner consistent with the Unicode Standard. Can also be specified via the embedded flag (?u).
Enable Unicode Predefined Character Classes	false	true false	Specifies conformance with the Unicode Technical Standard #18: Unicode Regular Expression Annex C: Compatibility Properties. Can also be specified via the embedded flag (?U).
Enable Unix Lines Mode	false	true false	Ind icates that only the ' +' line terminator is recognized in the behavior of '.', '^', and '$'. Can also be specified via the embedded flag (?d).
Include Capture Group 0	true	true false	Indicates that Capture Group 0 should be included as an attribute. Capture Group 0 represents the entirety of the regular expression match, is typically not used, and could have considerable length.
Enable repeating capture group	false	true false	If set to true, every string matching the capture groups will be extracted. Otherwise, if the Regular Expression matches more than once, only the first match will be extracted.

Dynamic Properties:

Dynamic Properties allow the user to specify both the name and value of a property.
Name Value Description
A FlowFile attribute A Regular Expression with one or more capturing group The first capture group, if any found, will be placed into that attribute name.But all capture groups, including the matching string sequence itself will also be provided at that attribute name with an index value provided.

Name	Value	Description
A FlowFile attribute	A Regular Expression with one or more capturing group	The first capture group, if any found, will be placed into that attribute name.But all capture groups, including the matching string sequence itself will also be provided at that attribute name with an index value provided.

Relationships:

Name	Description
unmatched	FlowFiles are routed to this relationship when no provided Regular Expression matches the content of the FlowFile
matched	FlowFiles are routed to this relationship when the Regular Expression is successfully evaluated and the FlowFile is modified as a result

Reads Attributes:

None specified.
Writes Attributes:
None specified.
State managem ent:
This component does not store state.
Restricted:
This component is not restricted.
Input requirement:
This component requires an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.FetchDistributedMapCache/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.FetchDistributedMapCache/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.FetchDistributedMapCache/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.FetchDistributedMapCache/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +FetchDistributedMapCache
FetchDistributedMapCache
Description:
Computes a cache key from FlowFile attributes, for each incoming FlowFile, and fetches the value from the Distributed Map Cache associated with that key. The incoming FlowFile's content is replaced with the binary data received by the Distributed Map Cache. If there is no value stored under that key then the flow file will be routed to 'not-found'. Note that the processor will always attempt to read the entire cached value into memory before placing it in it's destination. This could be potentially problematic if the cached value is very large.
Tags:
map, cache, fetch, distributed
Properties:
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
Name Default Value Allowable Values Description
Cache Entry Identifier ${hash.value} A FlowFile attribute, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the value used to identify duplicates; it is this value that is cached
Supports Expression Language: true
Distribu ted Cache Service Controller Service API:
DistributedMapCacheClient
Implementations: HBase_1_1_2_ClientMapCacheService
RedisDistributedMapCacheClientService
DistributedMapCacheClientService The Controller Service that is used to get the cached values.
Put Cache Value In Attribute If set, the cache value received will be put into an attr ibute of the FlowFile instead of a the content of theFlowFile. The attribute key to put to is determined by evaluating value of this property.
Supports Expression Language: true
Max Length To Put In Attribute 256 If routing the cache value to an attribute of the FlowFile (by setting the "Put Cache Value in attribute" property), the number of characters put to the attribute value will be at most this amount. This is important because attributes are held in memory and large attributes will quickly cause out of memory issues. If the output goes longer than this value, it will be truncated to fit. Consider making this smaller if able.
Character Set UTF-8 The Character Set in which the cached value is encoded. This will only be used when routing to an attribute.
Relationships:
Name Description
success If the cache was successfully communicated with it will be routed to this relationship
failure If unable to communicate with the cache or if the cache entry is evaluated to be blank, the FlowFile will be penalized and routed to this relationship
not-found If a FlowFile's Cache Entry Identifier was not found in the cache, it will be routed to this relationship
Reads Attributes:
None specified.
Writes Attributes:
Name Description
user-defined If the 'Put Cache Value In Attribute' property is set then whatever it is set to will become the attribute key and the value would be whatever the response was from the Distributed Map Cache.
State management:
This component d oes not store state.
Restricted:
This component is not restricted.
Input requirement:
This component requires an incoming relationship.
See Also:
DistributedMapCacheClientService, DistributedMapCacheServer, PutDistributedMapCache
\ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.FetchFTP/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.FetchFTP/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.FetchFTP/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.FetchFTP/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +FetchFTP
FetchFTP
Description:
Fetches the content of a file from a remote SFTP server and overwrites the contents of an incoming FlowFile with the content of the remote file.
Tags:
ftp, get, retrieve, files, fetch, remote, ingest, source, input
Properties:
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Language, a nd whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.
Name Default Value Allowable Values Description
Hostname The fully-qualified hostname or IP address of the host to fetch the data from
Supports Expression Language: true
Port 21 The port to connect to on the remote host to fetch the data from
Supports Expression Language: true
Username Username
Supports Expression Language: true
Password Password for the user account
Sensitive Property: true
Supports Expression Language: true
Remote File The fully qualified filename on the remote system
Supports Expression Language: true
Completion Strategy None
None
Move File
Delete File
Specifies what to do with the original file on the server once it has been pulled into NiFi. If the Completion Strategy fails, a warning will be logged but the data will still be transferred.
Move Destination Directory The directory on the remote server to the move the original file to once it has been ingested into NiFi. This property is ignored unless the Completion Strategy is set to "Move File". The specified directory must already exist onthe remote system, or the rename will fail.
Supports Expression Lang uage: true
Connection Timeout 30 sec Amount of time to wait before timing out while creating a connection
Data Timeout 30 sec When transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems
Use Compression false
true
false
Indicates whether or not ZLIB compression should be used when transferring files
Connection Mode Passive
Active
Passive
The FTP Connection Mode
Transfer Mode Binary
Binary
ASCII
The FTP Transfer Mode
Proxy Type DIRECT
DIRECT
HTTP
SOCKS
Proxy type used for file transfers
Proxy Host The fully qualified hostname or IP address of the proxy server
Proxy Port The port of the proxy server
Http Proxy Username Http Proxy Username
Http Proxy Password Http Proxy Password
Sensitive Property: true
Internal Buffer Size 16KB Set the internal buffer size for buffered data streams
Relationships:
Name Description
success All FlowFiles that are received are routed to success
comms.failure Any FlowFile that could not be fetched from the remote server due to a communications failure will be transferred to this Relationship.
not.found Any FlowFile for which we receive a 'Not Found' message from the remote server will be transferred to this Relationship.
permission.denied Any FlowFile that could not be fetched from the remote server due to insufficient permissions will be transferred to this Relationshi p.
Reads Attributes:
None specified.
Writes Attributes:
Name Description
ftp.remote.host The hostname or IP address from which the file was pulled
ftp.remote.port The port that was used to communicate with the remote FTP server
ftp.remote.filename The name of the remote file that was pulled
filename The filename is updated to point to the filename fo the remote file
path If the Remote File contains a directory name, that directory name will be added to the FlowFile using the 'path' attribute
State management:
This component does not store state.
Restricted:
This component is not restricted.
Input requirement:
This component requires an incoming relationship.
See Also:
GetSFTP, PutSFTP, GetFTP, PutFTP
\ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.FetchFile/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.FetchFile/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.FetchFile/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.FetchFile/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +FetchFile
FetchFile
Description:
Reads the contents of a file from disk and streams it into the contents of an incoming FlowFile. Once this is done, the file is optionally moved elsewhere or deleted to help keep the file system organized.
Tags:
local, files, filesystem, ingest, ingress, get, source, input, fetch, restricted
Properties:
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
Name Default Value Allowable Values Description
File to Fetch ${absolute.path}/${filename} The fully-qualified filename of the file to fetch from the file system
Supports Expression Language: true
Completion Strategy None
None
Move File
Delete File
Specifies what to do with the original file on the file system once it has been pulled into NiFi
Move Destination Directory The directory to the move the original file to once it has been fetched from the file system. This property is ignored unless the Completion Strategy is set to "Move File". If the directory does not exist, it will be created.
Supports Expression Language: true
Move Conflict Strategy Rename
Rename
Replace File
Keep Existing
Fail
If Completion Strategy is set to Move File and a file already exists in the destination directory with the same name, this property specifies how that naming conflict should be resolved
Log level when file not found ERROR
TRACE
DEBUG
INFO
WARN
ERROR
FATAL
NONE
Log level to use in case the file does not exist when the processor is triggered
Log level when permission denied ERROR
TRACE
DEBUG
INFO
WARN
ERROR
FATAL
NONE
Log level to use in case user jstorck does not have sufficient permissions to read the file
Relationships:
Name Description
success Any FlowFile that is successfully fetched from the file system will be transferred to this Relationship.
not.found Any FlowFile that could not be fetched from the file system because the file could not be found will be transferred to this Relationship.
permission.denied Any FlowFile that could not be fetched from the file system due to the user running NiFi not having sufficient permissions will be transferred to this Relationship.
failure Any FlowFile that could not be fetched from the file system for any reason other than insufficient permissions or the file not existing will be transferred to this Relationship.
Reads Attributes:
Non e specified.
Writes Attributes:
None specified.
State management:
This component does not store state.
Restricted:
Provides operator the ability to read from and delete any file that NiFi has access to.
Input requirement:
This component requires an incoming relationship.
See Also:
GetFile, PutFile, ListFile
\ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.FetchSFTP/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.FetchSFTP/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.FetchSFTP/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.FetchSFTP/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +FetchSFTP
FetchSFTP
Description:
Fetches the content of a file from a remote SFTP server and overwrites the contents of an incoming FlowFile with the content of the remote file.
Tags:
sftp, get, retrieve, files, fetch, remote, ingest, source, input
Properties:
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Language , and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.
Name Default Value Allowable Values Description
Hostname The fully-qualified hostname or IP address of the host to fetch the data from
Supports Expression Language: true
Port 22 The port to connect to on the remote host to fetch the data from
Supports Expression Language: true
Username Username
Supports Expression Language: true
Password Password for the user account
Sensitive Property: true
Supports Expression Language: true
Private Key Path The fully qualified path to the Private Key file
Supports Expression Language: true
Private Key Passphrase Password for the private key
Sensitive Property: true
Supports Expression Language: true
Remote File The fully qualified filename on the remote system
Supports Expression Language: true
Completion Strategy None
None
Move File
Delete File
Specifies what to do with the original file on the server once it has been pulled into NiFi. If the Completion Strategy fails, a warning w ill be logged but the data will still be transferred.
Move Destination Directory The directory on the remote server to the move the original file to once it has been ingested into NiFi. This property is ignored unless the Completion Strategy is set to "Move File". The specified directory must already exist onthe remote system, or the rename will fail.
Supports Expression Language: true
Connection Timeout 30 sec Amount of time to wait before timing out while creating a connection
Data Timeout 30 sec When transferring a file between the local and remote system, this value specifies how long is allowed to e lapse without any data being transferred between systems
Send Keep Alive On Timeout true
true
false
Indicates whether or not to send a single Keep Alive message when SSH socket times out
Host Key File If supplied, the given file will be used as the Host Key; otherwise, no use host key file will be used
Strict Host Key Checking false
true
false
Indicates whether or not strict enforcement of hosts keys should be applied
Use Compression false
true
false
< /td> Indicates whether or not ZLIB compression should be used when transferring files
Relationships:
Name Description
success All FlowFiles that are received are routed to success
comms.failure Any FlowFile that could not be fetched from the remote server due to a communications failure will be transferred to this Relationship.
not.found Any FlowFile for which we receive a 'Not Found' message from the remote server will be transferred to this Relationship.
permission.denied Any FlowFile that could not be fetched from the remote server due to insufficient permissions will be transferred to this Relationship.
Reads Attributes:
None specified.
Writes Attributes:
Name Description
sftp.remote.host Th e hostname or IP address from which the file was pulled
sftp.remote.port The port that was used to communicate with the remote SFTP server
sftp.remote.filename The name of the remote file that was pulled
filename The filename is updated to point to the filename fo the remote file
path If the Remote File contains a directory name, that directory name will be added to the FlowFile using the 'path' attribute
State management:
This component does not store state.
Restricted:
This component is not restricted.
Input requirement:
This component requires an incoming relationship.
See Also:
GetSFTP, PutSFTP, GetFTP, PutFTP
\ No newline at end of file

Name	Default Value	Allowable Values	Description
Cache Entry Identifier	${hash.value}		A FlowFile attribute, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the value used to identify duplicates; it is this value that is cached Supports Expression Language: true
Distribu ted Cache Service		Controller Service API: DistributedMapCacheClient Implementations: HBase_1_1_2_ClientMapCacheService RedisDistributedMapCacheClientService DistributedMapCacheClientService	The Controller Service that is used to get the cached values.
Put Cache Value In Attribute			If set, the cache value received will be put into an attr ibute of the FlowFile instead of a the content of theFlowFile. The attribute key to put to is determined by evaluating value of this property. Supports Expression Language: true
Max Length To Put In Attribute	256		If routing the cache value to an attribute of the FlowFile (by setting the "Put Cache Value in attribute" property), the number of characters put to the attribute value will be at most this amount. This is important because attributes are held in memory and large attributes will quickly cause out of memory issues. If the output goes longer than this value, it will be truncated to fit. Consider making this smaller if able.
Character Set	UTF-8		The Character Set in which the cached value is encoded. This will only be used when routing to an attribute.

Name	Description
success	If the cache was successfully communicated with it will be routed to this relationship
failure	If unable to communicate with the cache or if the cache entry is evaluated to be blank, the FlowFile will be penalized and routed to this relationship
not-found	If a FlowFile's Cache Entry Identifier was not found in the cache, it will be routed to this relationship

Name	Default Value	Allowable Values	Description
Hostname			The fully-qualified hostname or IP address of the host to fetch the data from Supports Expression Language: true
Port	21		The port to connect to on the remote host to fetch the data from Supports Expression Language: true
Username			Username Supports Expression Language: true
Password			Password for the user account Sensitive Property: true Supports Expression Language: true
Remote File			The fully qualified filename on the remote system Supports Expression Language: true
Completion Strategy	None	None Move File Delete File	Specifies what to do with the original file on the server once it has been pulled into NiFi. If the Completion Strategy fails, a warning will be logged but the data will still be transferred.
Move Destination Directory			The directory on the remote server to the move the original file to once it has been ingested into NiFi. This property is ignored unless the Completion Strategy is set to "Move File". The specified directory must already exist onthe remote system, or the rename will fail. Supports Expression Lang uage: true
Connection Timeout	30 sec		Amount of time to wait before timing out while creating a connection
Data Timeout	30 sec		When transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems
Use Compression	false	true false	Indicates whether or not ZLIB compression should be used when transferring files
Connection Mode	Passive	Active Passive	The FTP Connection Mode
Transfer Mode	Binary	Binary ASCII	The FTP Transfer Mode
Proxy Type	DIRECT	DIRECT HTTP SOCKS	Proxy type used for file transfers
Proxy Host			The fully qualified hostname or IP address of the proxy server
Proxy Port			The port of the proxy server
Http Proxy Username			Http Proxy Username
Http Proxy Password			Http Proxy Password Sensitive Property: true
Internal Buffer Size	16KB		Set the internal buffer size for buffered data streams

Name	Description
success	All FlowFiles that are received are routed to success
comms.failure	Any FlowFile that could not be fetched from the remote server due to a communications failure will be transferred to this Relationship.
not.found	Any FlowFile for which we receive a 'Not Found' message from the remote server will be transferred to this Relationship.
permission.denied	Any FlowFile that could not be fetched from the remote server due to insufficient permissions will be transferred to this Relationshi p.

Name	Description
ftp.remote.host	The hostname or IP address from which the file was pulled
ftp.remote.port	The port that was used to communicate with the remote FTP server
ftp.remote.filename	The name of the remote file that was pulled
filename	The filename is updated to point to the filename fo the remote file
path	If the Remote File contains a directory name, that directory name will be added to the FlowFile using the 'path' attribute

Name	Default Value	Allowable Values	Description
File to Fetch	${absolute.path}/${filename}		The fully-qualified filename of the file to fetch from the file system Supports Expression Language: true
Completion Strategy	None	None Move File Delete File	Specifies what to do with the original file on the file system once it has been pulled into NiFi
Move Destination Directory			The directory to the move the original file to once it has been fetched from the file system. This property is ignored unless the Completion Strategy is set to "Move File". If the directory does not exist, it will be created. Supports Expression Language: true
Move Conflict Strategy	Rename	Rename Replace File Keep Existing Fail	If Completion Strategy is set to Move File and a file already exists in the destination directory with the same name, this property specifies how that naming conflict should be resolved
Log level when file not found	ERROR	TRACE DEBUG INFO WARN ERROR FATAL NONE	Log level to use in case the file does not exist when the processor is triggered
Log level when permission denied	ERROR	TRACE DEBUG INFO WARN ERROR FATAL NONE	Log level to use in case user jstorck does not have sufficient permissions to read the file

Name	Description
success	Any FlowFile that is successfully fetched from the file system will be transferred to this Relationship.
not.found	Any FlowFile that could not be fetched from the file system because the file could not be found will be transferred to this Relationship.
permission.denied	Any FlowFile that could not be fetched from the file system due to the user running NiFi not having sufficient permissions will be transferred to this Relationship.
failure	Any FlowFile that could not be fetched from the file system for any reason other than insufficient permissions or the file not existing will be transferred to this Relationship.

Name	Description
sftp.remote.host	Th e hostname or IP address from which the file was pulled
sftp.remote.port	The port that was used to communicate with the remote SFTP server
sftp.remote.filename	The name of the remote file that was pulled
filename	The filename is updated to point to the filename fo the remote file
path	If the Remote File contains a directory name, that directory name will be added to the FlowFile using the 'path' attribute