Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GenerateFlowFile/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GenerateFlowFile/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GenerateFlowFile/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GenerateFlowFile/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +GenerateFlowFile

GenerateFlowFile

Description:

This processor creates FlowFiles with random data or custom content. GenerateFlowFile is usefulfor load testing, configuration, and simulation.

Tags:

test, random, generate

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

NameDefault ValueAllowable ValuesDescription
File Size0BThe size of the file that will be used
Batch Size1The number of FlowFiles to be transferred in each invocation
Data FormatText
  • Binary
  • Text
Specifies whether the data should be Text or Binary
Unique FlowFilesfalse
  • true
  • false
If true, each FlowFile that is generated will be unique. If false, a random value will be generated and all FlowFil es will get the same content but this offers much higher throughput
Custom TextIf Data Format is text and if Unique FlowFiles is false, then this custom text will be used as content of the generated FlowFiles and the File Size will be ignored. Finally, if Expression Language is used, evaluation will be performed only once per batch of generated FlowFiles
Supports Expression Language: true
Character SetUTF-8Specifies the character set to use when writing the bytes of Custom Text to a flow file.

Dynamic Properties:

Dynamic Properties allow the user to specify both the name and value of a property.
NameValueDescription
Generated FlowFile attribute nameGenerated FlowFile attribute valueSpecifies an attribute on generated FlowFiles defined by the Dynamic Property's key and value. If Expression Language is used, evaluation will be performed only once per batch of generated FlowFiles.
Supports Expression Language: true

Relationships:

NameDescription
success

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component does not allow an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GenerateTableFetch/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GenerateTableFetch/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GenerateTableFetch/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GenerateTableFetch/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1,6 @@ +GenerateTableFetch

GenerateTableFetch

Description:

Generates SQL select queries that fetch "pages" of rows from a table. The partition size property, along with the table's row count, determine the size and number of pages and generated FlowFiles. In addition, incremental fetching can be achieved by setting Maximum-Value Columns, which causes the processor to track the columns' maximum values, thus only fetching rows whose columns' values exceed the observed maximums. This processor is intended to be run on the Primary Node only. + +This processor can accept incoming connections; the behavior of the processor is different whether incoming connections are provided: + - If no incoming connection(s) are specified, the processor will generate SQL queries on the specified processor schedule. Expression Language is supported for many fields, but no flow file attributes are available. However the properties will be evaluated using the Variable Registry. + - If incoming connection(s) are specified and no flow file is available to a processor task, no work will be performed. + - If incoming connection(s) are specified and a flow file is available to a processor task, the flow file's attributes may be used in Expression Language for such fields as Table Name and others. However, the Max-Value Columns and Columns to Return fields must be empty or refer to columns that are available in each specified table.

Tags:

sql, select, jdbc, query, database, fetch, generate

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

NameDefault ValueAllowable ValuesDescription
Database Connection Pooling Service Controller Service API:
DBCPService
Implementations: DBCPConnectionPool
HiveConnectionPool
The Controller Service that is used to obtain a connection to the database.
Database TypeGeneric
  • Generic Generates ANSI SQL
  • Oracle Generates Oracle compliant SQL
  • Oracle 12+ Generates Oracle compliant SQL for version 12 or greater
  • MS SQL 2012+ Generates MS SQL Compatible SQL, for version 2012 or greater
  • MS SQL 2008 Generates MS SQL Compatible SQL for version 2008
The type/flavor of database, used for generating database-specific code. In many cases the Generic type should suffice, but some databases (such as Oracle) require custom SQL clauses.
Table NameThe name of the database table to be queried.
Supports Expression Language: true
Columns to ReturnA comma-separated list of column names to be used in the query. If your database requires special treatment of the names (quoting, e.g.), each name should include such treatment. If no column names are supplied, all columns in the specified table will be returned. NOTE: It is important to use consistent column names for a given table for incremental fetch to work properly.
Supports Expression Language: true
Maximum-value ColumnsA comma-separated list of column names. The processor will keep track of the maximum value for each column that has been returned since the processor started running. Using multiple columns implies an order to the column list, and each column's values are expected to increase more slowly than the previous columns' values. Thus, using multiple columns implies a hierarchical structure of columns, which is usually used for partitioning tables. This processor can be used to retrieve only those rows that have been added/updated since the last retrieval. Note that some JDBC types such as bit/boolean are not conducive to maintaining maximum value, so columns of these types should not be listed in this property, and will result in error(s) during processing. If no columns are provided, all rows from the table will be considered, which could have a performance impact. NOTE: It is important to use consistent max-value column names for a given table for incremental fetch to work properly.
Supports Expression Language: true
Max Wait Time0 secondsThe maximum amount of time allowed for a running SQL select query , zero means there is no limit. Max time less than 1 second will be equal to zero.
Supports Expression Lang uage: true
Partition Size10000The number of result rows to be fetched by each generated SQL statement. The total number of rows in the table divided by the partition size gives the number of SQL statements (i.e. FlowFiles) generated. A value of zero indicates that a single FlowFile is to be generated whose SQL statement will fetch all rows in the table.
Supports Expression Language: true
Additional WHERE clauseA custom clause to be added in the WHERE condition when generating SQL requests.
Supports Expression Language: true

Dynamic Properties:

Dynamic Properties allow the user to specify both the name and value of a property.
Name ValueDescription
Initial Max ValueAttribute Expression LanguageSpecifies an initial max value for max value columns. Properties should be added in the format `initial.maxvalue.{max_value_column}`.

Relationships:

NameDescription
successSuccessfully created FlowFile from SQL query result set.
failureThis relationship is only used when SQL query execution (using an incoming FlowFile) failed. The incoming FlowFile will be penalized and routed to this relationship. If no incoming connection(s) are specified, this relationship is unused.

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
generatetablefetch.sql.errorIf the processor has incoming connections, and processin g an incoming flow file causes a SQL Exception, the flow file is routed to failure and this attribute is set to the exception message.
generatetablefetch.tableNameThe name of the database table to be queried.
generatetablefetch.columnNamesThe comma-separated list of column names used in the query.
generatetablefetch.whereClauseWhere clause used in the query to get the expected rows.
generatetablefetch.maxColumnNamesThe comma-separated list of column names used to keep track of data that has been returned since the processor started running.
generatetablefetch.limitThe number of result rows to be fetched by the SQL statement.
generatetablefetch.offsetOffset to be used to retrieve the corresponding partition.

State management:

ScopeDescription
CLUSTER< /td>After performing a query on the specified table, the maximum values for the specified column(s) will be retained for use in future executions of the query. This allows the Processor to fetch only those records that have max values greater than the retained values. This can be used for incremental fetching, fetching of newly added rows, etc. To clear the maximum values, clear the state of the processor per the State Management documentation

Restricted:

This component is not restricted.

Input requirement:

This component allows an incoming relationship.

See Also:

QueryDatabaseTable, ExecuteSQL, ListDatabaseTables

\ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetFTP/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetFTP/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetFTP/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetFTP/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +GetFTP

GetFTP

Description:

Fetches files from an FTP Server and creates FlowFiles from them

Tags:

FTP, get, retrieve, files, fetch, remote, ingest, source, input

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Language, and whether a property is considered "sensitive", meaning that its value will be encr ypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
HostnameThe fully qualified hostname or IP address of the remote system
Supports Expression Language: true
Port21The port that the remote system is listening on for file transfers
Supports Expression Language: true
UsernameUsername
Supports Expression Language: true< /strong>
PasswordPassword for the user account
Sensitive Property: true
Supports Expression Language: true
Connection ModePassive
  • Active
  • Passive
The FTP Connection Mode
Transfer ModeBinary
  • Binary
  • ASCII
The FTP Transfer Mode
Remote PathThe path on the remote system from which to pull or push files
Supports Expression Language: true
File Filter RegexProvides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetched
Path Filter RegexWhen Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scanned
Polling Interval60 secDetermines how long to wait between fetching the listing for new files
Search Recursivelyfalse
  • true
  • false
If true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectories
Ignore Dotted Filestrue
  • true
  • false
If true, files whose names begin with a dot (".") will be ignored
Delete Originaltrue
  • true
  • false
Determines whether or not the file is deleted from the remote system after it has been successfully transferred
Connection Timeout30 secAmount of time to wait before timing out while creating a connection
Data Timeout30 secWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any da ta being transferred between systems
Max Selects100The maximum number of files to pull in a single connection
Remote Poll Batch Size5000The value specifies how many file paths to find in a given directory on the remote system when doing a file listing. This value in general should not need to be modified but when polling against a remote system with a tremendous number of files this value can be critical. Setting this value too high can result very poor performance and setting it too low can cause the flow to be slower than normal.
Use Natural Orderingfalse
  • true
  • false
If true, w ill pull files in the order in which they are naturally listed; otherwise, the order in which the files will be pulled is not defined
Proxy TypeDIRECT
  • DIRECT
  • HTTP
  • SOCKS
Proxy type used for file transfers
Proxy HostThe fully qualified hostname or IP address of the proxy server
Proxy PortThe port of the proxy server
Http Proxy UsernameHttp Proxy Username
Http Proxy PasswordHttp Proxy Password
Sensitive Property: true
Internal Buffer Size16KBSet the internal buffer size for buffered data streams
Use UTF-8 Encodingfalse
  • true
  • false
Tells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding.

Relationships:

NameDescription
successAll FlowFiles that are received are routed to success

Reads Attributes:

None specified.

Writes Attributes:

< tr>
NameDescription
filenameThe filename is set to the name of the file on the remote server
pathThe path is set to the path of the file's directory on the remote server. For example, if the <Remote Path> property is set to /tmp, files picked up from /tmp will have the path attribute set to /tmp. If the <Search Recursively> property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to /tmp/abc/1/2/3
file.lastModifiedTimeThe date and time that the source file was last modified
file.lastAccessTimeThe date and time that the file was last accessed. May not work on all file systems
file.ownerThe numeric owner id of the source file
file.groupThe numeric group id of the source file
file.permissionsThe read/write/execute permissions of the source file
absolute.pathThe full/absolute path from where a file was picked up. The current 'path' attribute is still populated, but may be a relative path

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component does not allow an incoming relationship.

See Also:

PutFTP

\ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetFile/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetFile/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetFile/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetFile/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +GetFile

GetFile

Description:

Creates FlowFiles from files in a directory. NiFi will ignore files it doesn't have at least read permissions for.

Tags:

local, files, filesystem, ingest, ingress, get, source, input, restricted

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

NameDefault ValueAllowable ValuesDescription
Input DirectoryThe input directory from which to pull files
Supports Expression Language: true
File Filter[^\.].*Only files whose names match the given regular expression will be picked up
Path FilterWhen Recurse Subdirectories is true, then only subdirectories whose path matches the given regular expression will be scanned
Batch Size10The maximum number of files to pull in each iterati on
Keep Source Filefalse
  • true
  • false
If true, the file is not deleted after it has been copied to the Content Repository; this causes the file to be picked up continually and is useful for testing purposes. If not keeping original NiFi will need write permissions on the directory it is pulling from otherwise it will ignore the file.
Recurse Subdirectoriestrue
  • true
  • false
Indicates whether or not to pull files from subdirectories
Polling Interval0 secIndicates how long to wait before performing a directory listing
Ignore Hi dden Filestrue
  • true
  • false
Indicates whether or not hidden files should be ignored
Minimum File Age0 secThe minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored
Maximum File AgeThe maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignored
Minimum File Size0 BThe minimum size that a file must be in order to be pulled
Maximum File SizeThe maximum size that a file can be in order to be pulled

Relationships:

NameDescription
successAll files are routed to success

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
filenameThe filename is set to the name of the file on disk
pathThe path is set to the relative path of the file's directory on disk. For example, if the <Input Directory> property is set to /tmp, files picked up from /tmp will have the path attribute set to ./. If the <Recurse Subdirectories> property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to abc/1/2/3
file .creationTimeThe date and time that the file was created. May not work on all file systems
file.lastModifiedTimeThe date and time that the file was last modified. May not work on all file systems
file.lastAccessTimeThe date and time that the file was last accessed. May not work on all file systems
file.ownerThe owner of the file. May not work on all file systems
file.groupThe group owner of the file. May not work on all file systems
file.permissionsThe read/write/execute permissions of the file. May not work on all file systems
absolute.pathThe full/absolute path from where a file was picked up. The current 'path' attribute is still populated, but may be a relative path

State management:

This component does not store state.

Restricted:

Provides operator the ability to read from and delet e any file that NiFi has access to.

Input requirement:

This component does not allow an incoming relationship.

See Also:

PutFile, FetchFile

\ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetHTTP/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetHTTP/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetHTTP/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetHTTP/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1,2 @@ +GetHTTP

GetHTTP

Description:

Fetches data from an HTTP or HTTPS URL and writes the data to the content of a FlowFile. Once the content has been fetched, the ETag and Last Modified dates are remembered (if the web server supports these concepts). This allows the Processor to fetch new data only if the remote data has changed or until the state is cleared. That is, once the content has been fetched from the given URL, it will not be fetched again until the content on the remote server changes. Note that due to limitations on state management, stored "last modified" and etag fields never expire. If the URL in GetHttp uses Expression Language that is unbounded, there is the potential for Out of Memory Errors to occur.

Tags:

get, fetch, poll, http, https, ingest, source, input

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Language, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
URLThe URL to pull from
Supports Expression Language: true
FilenameThe filename to assign to the file when pulled
Supports Expression Language: true
SSL Context ServiceController Service API:
SSLContextService
Implementations: StandardSSLContextService
StandardRestrictedSSLContextService
The Controller Service to use in order to obtain an SSL Context
Userna meUsername required to access the URL
PasswordPassword required to access the URL
Sensitive Property: true
Connection Timeout30 secHow long to wait when attempting to connect to the remote server before giving up
Data Timeout30 secHow long to wait between receiving segments of data from the remote server before giving up and discarding the partial file
User AgentWhat to report as the User Agent when we con nect to the remote server
Accept Content-TypeIf specified, requests will only accept the provided Content-Type
Follow Redirectsfalse
  • true
  • false
If we receive a 3xx HTTP Status Code from the server, indicates whether or not we should follow the redirect that the server specifies
Redirect Cookie Policydefault
  • default Default cookie policy that provides a higher degree of compatibility with common cookie management of popular HTTP agents for non-standard (Netscape style) cookies.
  • standard RFC 6265 compliant cookie policy (interoperability profile).
  • strict RFC 6265 compliant cookie policy (strict profile).
  • netscape Netscape draft compliant cookie policy.
  • ignore A cookie policy that ignores cookies.
When a HTTP server responds to a request with a redirect, this is the cookie policy used to copy cookies to the following request.
Proxy Hos tThe fully qualified hostname or IP address of the proxy server
Proxy PortThe port of the proxy server

Dynamic Properties:

Dynamic Properties allow the user to specify both the name and value of a property.
NameValueDescription
Header NameThe Expression Language to be used to populate the header valueThe additional headers to be sent by the processor whenever making a new HTTP request. + Setting a dynamic property name to XYZ and value to ${attribute} will result in the header 'XYZ: attribute_value' being sent to the HTTP endpoint

Relationships:

NameDescription
successAll files are transferred to the success relationship

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
filenameThe filename is set to the name of the file on the remote server
mime.typeThe MIME Type of the FlowFile, as reported by the HTTP Content-Type header

State management:

ScopeDescription
LOCALStores Last Modified Time and ETag headers returned by server so that the same data will not be fetched multiple times.

Restricted:

This compone nt is not restricted.

Input requirement:

This component does not allow an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetJMSQueue/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetJMSQueue/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetJMSQueue/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetJMSQueue/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +GetJMSQueue

GetJMSQueue

Deprecation notice:

This processor is deprecated and may be removed in future releases.

Please consider using one the following alternatives: ConsumeJMS

Description:

Pulls messages from a ActiveMQ JMS Queue, creating a FlowFile for each JMS Message or bundle of messages, as configured

Tags:

jms, queue, listen, get, pull, source, consume, consumer

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
JMS ProviderActiveMQ
  • ActiveMQ
The Provider used for the JMS Server
URLThe URL of the JMS Server
Destination NameThe name of the JMS Topic or queue to use
Communications Timeout30 secThe amount of time to wait when attempting to receive a message before giving up and assuming failure
Message Batch Size10The number of messages to pull/push in a single iteration of the processor
UsernameUsername used for authentication and authorization
PasswordPassword used for authentication and authorization
Sensitive Property: true
SSL Context ServiceController Service API:
SSLContextService
Implementations: StandardSSLContextService
StandardRestrictedSSLContextService
The Controller Service to use in order to obtain an SSL Context.
Acknowledgement ModeClient Acknowledge
  • Client Acknowledge
  • Auto Acknowledge
The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may provide better performance than Client Acknowledge.
Message SelectorThe JMS Message Selector to use in order to narrow the messages that are pulled
Copy JMS Properties to Attributestrue
  • true
  • false
Whether or not the JMS Message Properties should be copied to the FlowFile Attributes; if so, the attribute name will be jms.XXX, where XXX is the JMS Property name
Client ID PrefixA human-readable ID that can be used to associate connections with yourself so that the maintainers of the JMS Server know who to contact if problems arise

Relationships:

NameDescription
successAll FlowFiles are routed to success

Reads Attribu tes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component does not allow an incoming relationship.

See Also:

PutJMS

\ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetJMSTopic/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetJMSTopic/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetJMSTopic/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetJMSTopic/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +GetJMSTopic

GetJMSTopic

Deprecation notice:

This processor is deprecated and may be removed in future releases.

Please consider using one the following alternatives: ConsumeJMS

Description:

Pulls messages from a ActiveMQ JMS Topic, creating a FlowFile for each JMS Message or bundle of messages, as configured.

Tags:

jms, topic, subscription, durable, non-durable, listen, get, pull, source, consume, consumer

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
JMS ProviderActiveMQ
  • ActiveMQ
The Provider used for the JMS Server
URLThe URL of the JMS Server
Destination NameThe name of the JMS Topic or queue to use
Communications Timeout30 secThe amount of time to wait when attempting to receive a message before giving up and assuming failure
Message Batch Size10The number of messages to pull/push in a single iteration of the processor
UsernameUsername used for authentication and authorization
PasswordPassword used for authentication and authorization
Sensitive Property: true
SSL Context ServiceController Service API:
SSLContextService
Implementations: StandardSSLContextService
StandardRestrictedSSLContextService
The Controller Service to use in order to obtain an SSL Context.
Acknowledgement ModeClient Acknowledge
  • Client Acknowledge
  • Auto Acknowledge
The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may provide better performance than Client Acknowledge.
Me ssage SelectorThe JMS Message Selector to use in order to narrow the messages that are pulled
Copy JMS Properties to Attributestrue
  • true
  • false
Whether or not the JMS Message Properties should be copied to the FlowFile Attributes; if so, the attribute name will be jms.XXX, where XXX is the JMS Property name
Client ID PrefixA human-readable ID that can be used to associate connections with yourself so that the maintainers of the JMS Server know who to contact if problems arise
Use Durable Subscriptionfalse
  • true
  • false
If true, connections to the specified topic will use Durable Subscription so that messages are queued when we are not pulling them

Relationships:

NameDescription
successAll FlowFiles are routed to success

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component does not allow an incoming relationship.

See Also:

PutJMS

\ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetSFTP/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetSFTP/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetSFTP/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.GetSFTP/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +GetSFTP

GetSFTP

Description:

Fetches files from an SFTP Server and creates FlowFiles from them

Tags:

sftp, get, retrieve, files, fetch, remote, ingest, source, input

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Language, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
HostnameThe fully qualified hostname or IP address of the remote system
Supports Expression Language: true
Port22The port that the remote system is listening on for file transfers
Supports Expression Language: true
UsernameUsername
Supports Expression Language: t rue
PasswordPassword for the user account
Sensitive Property: true
Supports Expression Language: true
Private Key PathThe fully qualified path to the Private Key file
Supports Expression Language: true
Private Key PassphrasePassword for the private key
Sensitive Property: true
Supports Expression Language: true
Remote PathThe path on the remote system from which to pull or push files
Supports Expression Language: true
File Filter RegexProvides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetched
Path Filter RegexWhen Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scanned
Polling Interval60 secDetermines how long to wait between fetching the listing for new files
Search Recursivelyfalse
  • true
  • false
If true, will pull files from arbitrarily nes ted subdirectories; otherwise, will not traverse subdirectories
Ignore Dotted Filestrue
  • true
  • false
If true, files whose names begin with a dot (".") will be ignored
Delete Originaltrue
  • true
  • false
Determines whether or not the file is deleted from the remote system after it has been successfully transferred
Connection Timeout30 secAmount of time to wait before timing out while creating a connection
Data Timeout30 secWhen transferr ing a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems
Host Key FileIf supplied, the given file will be used as the Host Key; otherwise, no use host key file will be used
Max Selects100The maximum number of files to pull in a single connection
Remote Poll Batch Size5000The value specifies how many file paths to find in a given directory on the remote system when doing a file listing. This value in general should not need to be modified but when polling against a remote system with a tremendous number of files this value can be critical. Setting this value too high can result very poor performance and setting it too low can cause the flow to be slower than normal.
Strict Host Key Checkingfalse
  • true
  • false
Indicates whether or not strict enforcement of hosts keys should be applied
Send Keep Alive On Timeouttrue
  • true
  • false
Indicates whether or not to send a single Keep Alive message when SSH socket times out
Use Compressionfalse
  • true
  • false
Indicates whether or not ZLIB compression should be used when transferring files
Use Natural Orderingfalse
  • true
  • false
If true, will pull files in the order in which they are naturally listed; otherwise, the order in which the files will be pulled is not defined

Relationships:

NameDescription
successAll FlowFiles that are received are routed to success

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
filenameThe filename is set to the name of the file on the remote server
pathThe path is set to the path of the file's directory on the remote server. For example, if the <Remote Path> property is set to /tmp, files picked up from /tmp will have the path attribute set to /tmp. If the <Search Recursivel y> property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to /tmp/abc/1/2/3
file.lastModifiedTimeThe date and time that the source file was last modified
file.ownerThe numeric owner id of the source file
file.groupThe numeric group id of the source file
file.permissionsThe read/write/execute permissions of the source file
absolute.pathThe full/absolute path from where a file was picked up. The current 'path' attribute is still populated, but may be a relative path

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component does not allow an incoming relationship.

See Also:

PutSFTP

\ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HandleHttpRequest/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HandleHttpRequest/additionalDetails.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HandleHttpRequest/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HandleHttpRequest/additionalDetails.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1,43 @@ + + + + + + HandleHttpRequest + + + + +

Usage Description

+

+ The pairing of this Processor with a HandleHttpResponse Processor + provides the ability to use NiFi to visually construct a web server that can carry out any functionality that is available + through the existing Processors. For example, one could construct a Web-based front end to an SFTP Server by constructing a + flow such as: +

+ +

+ HandleHttpRequest -> + PutSFTP -> + HandleHttpResponse +

+ +

+ The HandleHttpRequest Processor provides several Properties to configure which methods are supported, the paths that are + supported, and SSL configuration. +

+ + Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HandleHttpRequest/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HandleHttpRequest/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HandleHttpRequest/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HandleHttpRequest/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +HandleHttpRequest

HandleHttpRequest

Description:

Starts an HTTP Server and listens for HTTP Requests. For each request, creates a FlowFile and transfers to 'success'. This Processor is designed to be used in conjunction with the HandleHttpResponse Processor in order to create a Web Service

Additional Details...

Tags:

http, https, request, listen, ingress, web service

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. Th e table also indicates any default values.

NameDefault ValueAllowable ValuesDescription
Listening Port80The Port to listen on for incoming HTTP requests
HostnameThe Hostname to bind to. If not specified, will bind to all hosts
SSL Context ServiceController Service API:
RestrictedSSLContextService
Implementation: StandardRestrictedSSLContextService
The SSL Context Service to use in order to secure the server. If s pecified, the server will accept only HTTPS requests; otherwise, the server will accept only HTTP requests
HTTP Context MapController Service API:
HttpContextMap
Implementation: StandardHttpContextMap
The HTTP Context Map Controller Service to use for caching the HTTP Request Information
Allowed PathsA Regular Expression that specifies the valid HTTP Paths that are allowed in the incoming URL Requests. If this value is specified and the path of the HTTP Requests does not match this Regular Expression, the Processor will respond with a 404: NotFound
Default URL Ch aracter SetUTF-8The character set to use for decoding URL parameters if the HTTP Request does not supply one
Allow GETtrue
  • true
  • false
Allow HTTP GET Method
Allow POSTtrue
  • true
  • false
Allow HTTP POST Method
Allow PUTtrue
  • true
  • false
Allow HTTP PUT Method
Allow DELETEtrue
  • true
  • false
Allow HTTP DELETE Method
Allow HEADfalse
  • true
  • false
Allow HTTP HEAD Method
Allow OPTIONSfalse
  • true
  • false
Allow HTTP OPTIONS Method
Additional HTTP MethodsA comma-separated list of non-standard HTTP Methods that should be allowed
Client AuthenticationNo Authentication
  • No Authentication Processor will not authenticate clients. Anyone can communicate with this Processor anonymously
  • Want Authentication Processor will try to verify the client but if unable to verify will allow the client to communicate anonymously
  • Need Authentication Processor will reject communications from any client unless the client provides a certificate that is trusted by the TrustStorespecified in the SSL Context Service
Specifies whether or not the Processor should authenticate clients. This value is ignored if the <SSL Context Service> Property is n ot specified or the SSL Context provided uses only a KeyStore and not a TrustStore.
Container Queue Size50The size of the queue for Http Request Containers

Relationships:

NameDescription
successAll content that is received is routed to the 'success' relationship

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
http.context.identifierAn identifier that allows the HandleHttpRequest and HandleHttpResponse to coordinate which FlowFile belongs to which HTTP Request/Response.
mime.typeThe MIME Type of the data, according to the HTTP Header "Content-Type"
http.servlet.pathThe part of the reques t URL that is considered the Servlet Path
http.context.pathThe part of the request URL that is considered to be the Context Path
http.methodThe HTTP Method that was used for the request, such as GET or POST
http.local.nameIP address/hostname of the server
http.server.portListening port of the server
http.query.stringThe query string portion of the Request URL
http.remote.hostThe hostname of the requestor
http.remote.addrThe hostname:port combination of the requestor
http.remote.userThe username of the requestor
http.protocolThe protocol used to communicate
http.request.uriThe full Request URL
http.auth.typeThe type of HTTP Authorization used
http.principal.nameThe name of the authenticated user making the request
http.subject.dnThe Distinguished Name of the requestor. This value will not be populated unless the Processor is configured to use an SSLContext Service
http.issuer.dnThe Distinguished Name of the entity that issued the Subject's certificate. This value will not be populated unless the Processor is configured to use an SSLContext Service
http.headers.XXXEach of the HTTP Headers that is received in the request will be added as an attribute, prefixed with "http.headers." For example, if the request contains an HTTP Header named "x-my-header", then the value will be added to an attribute named "http.headers.x-my-header"

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component does not allow an incoming relationship.

See Also:

HandleHttpResponse, StandardHttpContextMap,

\ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HandleHttpResponse/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HandleHttpResponse/additionalDetails.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HandleHttpResponse/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HandleHttpResponse/additionalDetails.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1,44 @@ + + + + + + HandleHttpResponse + + + + + +

Usage Description:

+

+ The pairing of this Processor with a HandleHttpRequest Processor + provides the ability to use NiFi to visually construct a web server that can carry out any functionality that is available + through the existing Processors. For example, one could construct a Web-based front end to an SFTP Server by constructing a + flow such as: +

+ +

+ HandleHttpRequest -> + PutSFTP -> + HandleHttpResponse +

+ +

+ This Processor must be configured with the same <HTTP Context Map> service as the corresponding HandleHttpRequest Processor. + Otherwise, all FlowFiles will be routed to the 'failure' relationship. +

+ + Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HandleHttpResponse/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HandleHttpResponse/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HandleHttpResponse/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HandleHttpResponse/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +HandleHttpResponse

HandleHttpResponse

Description:

Sends an HTTP Response to the Requestor that generated a FlowFile. This Processor is designed to be used in conjunction with the HandleHttpRequest in order to create a web service.

Additional Details...

Tags:

http, https, response, egress, web service

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property s upports the NiFi Expression Language.

< /table>

Dynamic Properties:

Dynamic Properties allow the user to specify both the name and value of a property.

NameDefault ValueAllowable ValuesDescription
HTTP Status CodeThe HTTP Status Code to use when responding to the HTTP Request. See Section 10 of RFC 2616 for more information.
Supports Expression Language: true
HTTP Context MapController Service API:
HttpContextMap
Implementation: StandardHttpContextMap
The HTTP Context Map Controller Service to use for caching the HTTP Request Information
NameValueDescription
An HTTP header nameAn HTTP header valueThese HTTPHeaders are set in the HTTP Response

Relationships:

NameDescription
successFlowFiles will be routed to this Relationship after the response has been successfully sent to the requestor
failureFlowFiles will be routed to this Relationship if the Processor is unable to respond to the requestor. This may happen, for instance, if the connection times out or if NiFi is restarted before responding to the HTTP Request.

Reads Attributes:

< td>The value of this attribute is used to lookup the HTTP Response so that the proper message can be sent back to the requestor. If this attribute is missing, the FlowFile will be routed to 'failure.'
NameDescription
http.context.identifier
http.request.uriValue of the URI requested by the client. Used for provenance event.
http.remote.hostIP address of the client. Used for provenance event.
http.local.nameIP address/hostname of the server. Used for provenance event.
http.server.portListening port of the server. Used for provenance event.
http.subject.dnSSL distinguished name (if any). Used for provenance event.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

See Also:

HandleHttpRequest, StandardHttpContextMap, StandardSSLContextService

\ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HashAttribute/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HashAttribute/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HashAttribute/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache.nifi.processors.standard.HashAttribute/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +HashAttribute

HashAttribute

Description:

Hashes together the key/value pairs of several FlowFile Attributes and adds the hash as a new attribute. Optional properties are to be added such that the name of the property is the name of a FlowFile Attribute to consider and the value of the property is a regular expression that, if matched by the attribute value, will cause that attribute to be used as part of the hash. If the regular expression contains a capturing group, only the value of the capturing group will be used.

Tags:

attributes, hash

Properties:

In the list b elow, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values.

NameDefault ValueAllowable ValuesDescription
Hash Value Attribute KeyThe name of the FlowFile Attribute where the hash value should be stored

Dynamic Properties:

Dynamic Properties allow the user to specify both the name and value of a property.
NameValueDescription
A flowfile attribute key for attribute inspectionA Regular ExpressionThis regular expression is evaluated against the flowfile attribute values. If the regular expression contains a capturing group, the value of that group will be used when comparing flow file attributes. Otherwise, the original flow file attribute's value will be used if and only if the value matches the given regular expression.

Relationships:

NameDescription
successUsed for FlowFiles that have a hash value added
failureUsed for FlowFiles that are missing required attributes

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
<Hash Value Attribute Key>This Processor adds an attribute whose value is the result of Hashing the existing FlowFile attributes. The name of this attribute is specified by the <Hash Value Attribute Key> property.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Inpu t requirement:

This component requires an incoming relationship. \ No newline at end of file