Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-5-nar/1.4.0/org.apache.nifi.processors.elasticsearch.PutElasticsearch5/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-5-nar/1.4.0/org.apache.nifi.processors.elasticsearch.PutElasticsearch5/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-5-nar/1.4.0/org.apache.nifi.processors.elasticsearch.PutElasticsearch5/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-5-nar/1.4.0/org.apache.nifi.processors.elasticsearch.PutElasticsearch5/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +PutElasticsearch5

PutElasticsearch5

Description:

Writes the contents of a FlowFile to Elasticsearch, using the specified parameters such as the index to insert into and the type of the document. If the cluster has been configured for authorization and/or secure transport (SSL/TLS), and the X-Pack plugin is available, secure connections can be made. This processor supports Elasticsearch 5.x clusters.

Tags:

elasticsearch, elasticsearch 5, insert, update, write, put

Properties:

In the list below, the names of required properties appear in bold. Any oth er properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Language, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
Cluster NameelasticsearchName of the ES cluster (for example, elasticsearch_brew). Defaults to 'elasticsearch'
Supports Expression Language: true
ElasticSearch HostsElasticSearch Hosts, which should be comma separated and colon for hostname/port host1:port,host2:port,.... For example testcluster:9300. This processor uses the Transport Client to connect to hosts. The default transport client port is 9300.
Supports Expression Language: true
SSL Context ServiceController Service API:
SSLContextService
Implementations: StandardSSLContextService
StandardRestrictedSSLContextService
The SSL Context Service used to provide client certificate information for TLS/SSL connections. This service only app lies if the Elasticsearch endpoint(s) have been secured with TLS/SSL.
X-Pack Transport LocationSpecifies the path to the JAR(s) for the Elasticsearch X-Pack Transport feature. If the Elasticsearch cluster has been secured with the X-Pack plugin, then the X-Pack Transport JARs must also be available to this processor. Note: Do NOT place the X-Pack JARs into NiFi's lib/ directory, doing so will prevent the X-Pack Transport JARs from being loaded.
Supports Expression Language: true
UsernameUsername to access the Elasticsearch cluster
Supports Expression Language: true
PasswordPassword to access the Elasticsearch cl uster
Sensitive Property: true
ElasticSearch Ping Timeout5sThe ping timeout used to determine when a node is unreachable. For example, 5s (5 seconds). If non-local recommended is 30s
Supports Expression Language: true
Sampler Interval5sHow often to sample / ping the nodes listed and connected. For example, 5s (5 seconds). If non-local recommended is 30s.
Supports Expression Language: true
Identifier AttributeThe name of the attribute containing the identifier for each FlowFile
IndexThe name of the index to insert into
Supports Expression Language: true
TypeThe type of this document (used by Elasticsearch for indexing and searching)
Supports Expression Language: true
Character SetUTF-8Specifies the character set of the document data.
Supports Expression Language: true
Batch Size100The preferred number of FlowFiles to put to the database in a single transaction
Supports Expression Language: true
In dex OperationindexThe type of the operation used to index (index, update, upsert)
Supports Expression Language: true

Relationships:

NameDescription
retryA FlowFile is routed to this relationship if the database cannot be updated but attempting the operation again may succeed
successAll FlowFiles that are written to Elasticsearch are routed to this relationship
failureAll FlowFiles that cannot be written to Elasticsearch are routed to this relationship

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incom ing relationship.

See Also:

FetchElasticsearch5, PutElasticsearch5

\ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.FetchElasticsearch/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.FetchElasticsearch/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.FetchElasticsearch/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.FetchElasticsearch/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +FetchElasticsearch

FetchElasticsearch

Description:

Retrieves a document from Elasticsearch using the specified connection properties and the identifier of the document to retrieve. If the cluster has been configured for authorization and/or secure transport (SSL/TLS) and the Shield plugin is available, secure connections can be made. This processor supports Elasticsearch 2.x clusters.

Tags:

elasticsearch, fetch, read, get

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Language, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
Cluster NameelasticsearchName of the ES cluster (for example, elasticsearch_brew). Defaults to 'elasticsearch'
Supports Expression Language: true
ElasticSearch Hosts ElasticSearch Hosts, which should be comma separated and colon for hostname/port host1:port,host2:port,.... For example testcluster:9300. This processor uses the Transport Client to connect to hosts. The default transport client port is 9300.
Supports Expression Language: true
SSL Context ServiceController Service API:
SSLContextService
Implementations: StandardSSLContextService
StandardRestrictedSSLContextService
The SSL Context Service used to provide client certificate information for TLS/SSL connections. This service only applies if the Elasticsearch endpoint(s) have been secured with TLS/SSL.
Shield Plugin FilenameSpecifies the path to the JAR for the Elasticsearch Shield plugin. If the Elasticsearch cluster has been secured with the Shield plugin, then the Shield plugin JAR must also be available to this processor. Note: Do NOT place the Shield JAR into NiFi's lib/ directory, doing so will prevent the Shield plugin from being loaded.
Supports Expression Language: true
UsernameUsername to access the Elasticsearch cluster
Supports Expression Language: true
PasswordPassword to access the Elasticsearch cluster
Sensitive Property: true
Supports Expression Language: true
ElasticSearch Ping Timeout5sThe ping timeout used to determine when a node is unreachable. For example, 5s (5 seconds). If non-local recommended is 30s
Supports Expression Language: true
Sampler Interval5sHow often to sample / ping the nodes listed and connected. For example, 5s (5 seconds). If non-local recommended is 30s.
Supports Expression Language: true
Document IdentifierThe identifier for the document to be fetched
Supports Expression Language: true
Index< /td>The name of the index to read from
Supports Expression Language: true
TypeThe type of this document (used by Elasticsearch for indexing and searching)
Supports Expression Language: true
Character SetUTF-8Specifies the character set of the document data.
Supports Expression Language: true

Relationships:

NameDescription
retryA FlowFile is routed to this relationship if the document cannot be fetched but attempting the operation again may succeed
successAll FlowFiles th at are read from Elasticsearch are routed to this relationship
failureAll FlowFiles that cannot be read from Elasticsearch are routed to this relationship
not foundA FlowFile is routed to this relationship if the specified document does not exist in the Elasticsearch cluster

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
filenameThe filename attributes is set to the document identifier
es.indexThe Elasticsearch index containing the document
es.typeThe Elasticsearch document type

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.FetchElasticsearchHttp/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.FetchElasticsearchHttp/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.FetchElasticsearchHttp/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.FetchElasticsearchHttp/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +FetchElasticsearchHttp

FetchElasticsearchHttp

Description:

Retrieves a document from Elasticsearch using the specified connection properties and the identifier of the document to retrieve. Note that the full body of the document will be read into memory before being written to a Flow File for transfer.

Tags:

elasticsearch, fetch, read, get, http

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property sup ports the NiFi Expression Language, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
Elasticsearch URLElasticsearch URL which will be connected to, including scheme (http, e.g.), host, and port. The default port for the REST API is 9200.
Supports Expression Language: true
SSL Context ServiceController Service API:
SSLContextService
Imp lementations: StandardSSLContextService
StandardRestrictedSSLContextService
The SSL Context Service used to provide client certificate information for TLS/SSL connections. This service only applies if the Elasticsearch endpoint(s) have been secured with TLS/SSL.
UsernameUsername to access the Elasticsearch cluster
Supports Expression Language: true
PasswordPassword to access the Elasticsearch cluster
Sensitive Property: true
Supports Exp ression Language: true
Connection Timeout5 secsMax wait time for the connection to the Elasticsearch REST API.
Supports Expression Language: true
Response Timeout15 secsMax wait time for a response from the Elasticsearch REST API.
Supports Expression Language: true
Document IdentifierThe identifier of the document to be fetched
Supports Expression Language: true
IndexThe name of the index to read from.
Supports Expression Language: true
TypeThe (optional) type of this document, used by Elasticsearch for indexing and searching. If the property is empty, the first document matching the identifier across all types will be retrieved.
Supports Expression Language: true
FieldsA comma-separated list of fields to retrieve from the document. If the Fields property is left blank, then the entire document's source will be retrieved.
Supports Expression Language: true

Dynamic Properties:

Dynamic Properties allow the user to specify both the name and value of a property.
NameValueDescription
A URL query p arameterThe value to set it toAdds the specified property name/value as a query parameter in the Elasticsearch URL used for processing
Supports Expression Language: true

Relationships:

NameDescription
retryA FlowFile is routed to this relationship if the document cannot be fetched but attempting the operation again may succeed. Note that if the processor has no incoming connections, flow files may still be sent to this relationship based on the processor properties and the results of the fetch operation.
successAll FlowFiles that are read from Elasticsearch are routed to this relationship.
failureAll FlowFiles that cannot be read from Elasticsearch are routed to this relationship. Note that only incoming flow files will be routed to failure.
not foundA FlowFil e is routed to this relationship if the specified document does not exist in the Elasticsearch cluster. Note that if the processor has no incoming connections, flow files may still be sent to this relationship based on the processor properties and the results of the fetch operation.

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
filenameThe filename attribute is set to the document identifier
es.indexThe Elasticsearch index containing the document
es.typeThe Elasticsearch document type

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component allows an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.PutElasticsearch/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.PutElasticsearch/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.PutElasticsearch/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.PutElasticsearch/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +PutElasticsearch

PutElasticsearch

Description:

Writes the contents of a FlowFile to Elasticsearch, using the specified parameters such as the index to insert into and the type of the document. If the cluster has been configured for authorization and/or secure transport (SSL/TLS) and the Shield plugin is available, secure connections can be made. This processor supports Elasticsearch 2.x clusters.

Tags:

elasticsearch, insert, update, write, put

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not i n bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Language, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
Cluster NameelasticsearchName of the ES cluster (for example, elasticsearch_brew). Defaults to 'elasticsearch'
Supports Expression Language: true
ElasticSearch HostsElasticSearch Hosts, which should be comma separated and colon for hostname/port host1:port,host2:port,.... For example testcluster:9300. This processor uses the Transport Client to connect to hosts. The default transport client port is 9300.
Supports Expression Language: true
SSL Context ServiceController Service API:
SSLContextService
Implementations: StandardSSLContextService
StandardRestrictedSSLContextService
The SSL Context Service used to provide client certificate information for TLS/SSL connections. This service only applies if the Elastics earch endpoint(s) have been secured with TLS/SSL.
Shield Plugin FilenameSpecifies the path to the JAR for the Elasticsearch Shield plugin. If the Elasticsearch cluster has been secured with the Shield plugin, then the Shield plugin JAR must also be available to this processor. Note: Do NOT place the Shield JAR into NiFi's lib/ directory, doing so will prevent the Shield plugin from being loaded.
Supports Expression Language: true
UsernameUsername to access the Elasticsearch cluster
Supports Expression Language: true
PasswordPassword to access the Elasticsearch cluster
Sensitive Property: true
Supports Expression Language: true
ElasticSearch Ping Timeout5sThe ping timeout used to determine when a node is unreachable. For example, 5s (5 seconds). If non-local recommended is 30s
Supports Expression Language: true
Sampler Interval5sHow often to sample / ping the nodes listed and connected. For example, 5s (5 seconds). If non-local recommended is 30s.
Supports Expression Language: true
Identifier AttributeThe name of the attribute containing the identifier for each FlowFile
IndexThe name of the index to insert into
Supports Expression Language: true
TypeThe type of this document (used by Elasticsearch for indexing and searching)
Supports Expression Language: true
Character SetUTF-8Specifies the character set of the document data.
Supports Expression Language: true
Batch Size100The preferred number of FlowFiles to put to the database in a single transaction
Supports Expression Language: true
Index OperationindexThe type of the operation used to index (index, update, upsert)
Supports Expression Language: true

Relationships:

NameDescription
retryA FlowFile is routed to this relationship if the database cannot be updated but attempting the operation again may succeed
successAll FlowFiles that are written to Elasticsearch are routed to this relationship
failureAll FlowFiles that cannot be written to Elasticsearch are routed to this relationship

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.PutElasticsearchHttp/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.PutElasticsearchHttp/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.PutElasticsearchHttp/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.PutElasticsearchHttp/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +PutElasticsearchHttp

PutElasticsearchHttp

Description:

Writes the contents of a FlowFile to Elasticsearch, using the specified parameters such as the index to insert into and the type of the document.

Tags:

elasticsearch, insert, update, upsert, delete, write, put, http

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi E xpression Language, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
Elasticsearch URLElasticsearch URL which will be connected to, including scheme (http, e.g.), host, and port. The default port for the REST API is 9200.
Supports Expression Language: true
SSL Context ServiceController Service API:
SSLContextService
Implementations: StandardSSLContextService
StandardRestrictedSSLContextService
The SSL Context Service used to provide client certificate information for TLS/SSL connections. This service only applies if the Elasticsearch endpoint(s) have been secured with TLS/SSL.
UsernameUsername to access the Elasticsearch cluster
Supports Expression Language: true
PasswordPassword to access the Elasticsearch cluster
Sensitive Property: true
Supports Expression Language: true
Connection Timeout5 secsMax wait time for the connection to the Elasticsearch REST API.
Supports Expression Language: true
Response Timeout15 secsMax wait time for a response from the Elasticsearch REST API.
Supports Expression Language: true
Identifier AttributeThe name of the FlowFile attribute containing the identifier for the document. If the Index Operation is "index", this property may be left empty or evaluate to an empty value, in which case the document's identifier will be auto-generated by Elasticsearch. For all other Index Operations, the attribute must evaluate to a non-empty value.
IndexThe name of the index to insert into
Supports Expression Language: true
TypeThe type of this document (used by Elasticsearch for indexing and searching)
Supports Expression Language: true
Character SetUTF-8Specifies the character set of the document data.
Supports Expression Language: true
Batch Size100The preferred number of flow files to put to the database in a single transaction. Note that the contents of the flow files will be stor ed in memory until the bulk operation is performed. Also the results should be returned in the same order the flow files were received.
Supports Expression Language: true
Index OperationindexThe type of the operation used to index (index, update, upsert, delete)
Supports Expression Language: true

Dynamic Properties:

Dynamic Properties allow the user to specify both the name and value of a property.
NameValueDescription
A URL query parameterThe value to set it toAdds the specified property name/value as a query parameter in the Elasticsearch URL used for processing
Supports Expression Language: true

Relationships:

NameDescription
retryA FlowFile is routed to this relationship if the database cannot be updated but attempting the operation again may succeed
successAll FlowFiles that are written to Elasticsearch are routed to this relationship
failureAll FlowFiles that cannot be written to Elasticsearch are routed to this relationship

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.PutElasticsearchHttpRecord/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.PutElasticsearchHttpRecord/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.PutElasticsearchHttpRecord/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.PutElasticsearchHttpRecord/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +PutElasticsearchHttpRecord

PutElasticsearchHttpRecord

Description:

Writes the records from a FlowFile into to Elasticsearch, using the specified parameters such as the index to insert into and the type of the document, as well as the operation type (index, upsert, delete, etc.). Note: The Bulk API is used to send the records. This means that the entire contents of the incoming flow file are read into memory, and each record is transformed into a JSON document which is added to a single HTTP request body. For very large flow files (files with a large number of records, e.g.), this could cause mem ory usage issues.

Tags:

elasticsearch, insert, update, upsert, delete, write, put, http, record

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Language, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
Elasticsearch URLElasticsearch URL which wi ll be connected to, including scheme (http, e.g.), host, and port. The default port for the REST API is 9200.
Supports Expression Language: true
SSL Context ServiceController Service API:
SSLContextService
Implementations: StandardSSLContextService
StandardRestrictedSSLContextService
The SSL Context Service used to provide client certificate information for TLS/SSL connections. This service only applies if the Elasticsearch endpoint(s) have been secured with TLS/SSL.
UsernameUsername to access the Elasticsearch cluster
Supports Expression Language: true
PasswordPassword to access the Elasticsearch cluster
Sensitive Property: true
Supports Expression Language: true
Connection Timeout5 secsMax wait time for the connection to the Elasticsearch REST API.
Supports Expression Language: true
Response Timeout15 secsMax wait time for a response from the Elasticsearch REST API.
Supports Expression Language: true
Record ReaderController Service API:
RecordReaderFactory
Implementations: CSVReader
GrokReader
AvroReader
JsonTreeReader
JsonPathReader
ScriptedReader
Specifies the Controller Service to use for parsing incoming d ata and determining the data's schema.
Identifier Record PathA RecordPath pointing to a field in the record(s) that contains the identifier for the document. If the Index Operation is "index", this property may be left empty or evaluate to an empty value, in which case the document's identifier will be auto-generated by Elasticsearch. For all other Index Operations, the field's value must be non-empty.
Supports Expression Language: true
IndexThe name of the index to insert into
Supports Expression Language: true
TypeThe type of this document (used by Elasticsearch for indexing a nd searching)
Supports Expression Language: true
Index OperationindexThe type of the operation used to index (index, update, upsert, delete)
Supports Expression Language: true

Dynamic Properties:

Dynamic Properties allow the user to specify both the name and value of a property.
NameValueDescription
A URL query parameterThe value to set it toAdds the specified property name/value as a query parameter in the Elasticsearch URL used for processing
Supports Expression Language: true

Relationships:

NameDescription
retryA FlowFile is routed to this relationship if the database cannot be updated but attempting the operation again may succeed
successAll FlowFiles that are written to Elasticsearch are routed to this relationship
failureAll FlowFiles that cannot be written to Elasticsearch are routed to this relationship

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.QueryElasticsearchHttp/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.QueryElasticsearchHttp/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.QueryElasticsearchHttp/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.QueryElasticsearchHttp/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +QueryElasticsearchHttp

QueryElasticsearchHttp

Description:

Queries Elasticsearch using the specified connection properties. Note that the full body of each page of documents will be read into memory before being written to Flow Files for transfer. Also note that the Elasticsearch max_result_window index setting is the upper bound on the number of records that can be retrieved using this query. To retrieve more records, use the ScrollElasticsearchHttp processor.

Tags:

elasticsearch, query, read, get, http

Properties:

In the list below, the names of required propert ies appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Language, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
Elasticsearch URLElasticsearch URL which will be connected to, including scheme (http, e.g.), host, and port. The default port for the REST API is 9200.
Supports Expression Language: true
SSL Context ServiceController Service API:
SSLContextService
Implementations: StandardSSLContextService
StandardRestrictedSSLContextService
The SSL Context Service used to provide client certificate information for TLS/SSL connections. This service only applies if the Elasticsearch endpoint(s) have been secured with TLS/SSL.
UsernameUsername to access the Elasticsearch cluster
Supports Expression Language: true
PasswordPassword to access the Elasticsearch cluster
Sensitive Property: true
Supports Expression Language: true
Connection Timeout5 secsMax wait time for the connection to the Elasticsearch REST API.
Supports Expression Language: true
Response Timeout15 secsMax wait time for a response from the Elasticsearch REST API.
Supports Expression Language: true
QueryThe Lucene-style query to run against ElasticSearch (e.g., genre:blues AND -artist:muddy)
Supports Expression La nguage: true
Page Size20Determines how many documents to return per page during scrolling.
Supports Expression Language: true
IndexThe name of the index to read from. If the property is set to _all, the query will match across all indexes.
Supports Expression Language: true
TypeThe (optional) type of this query, used by Elasticsearch for indexing and searching. If the property is empty, the the query will match across all types.
Supports Expression Language: true
FieldsA comma-separated list of fields to retrieve from the document. If the Fields property is left blank, then the entire document's source will be retrieved.
Supports Expression Language: true
SortA sort parameter (e.g., timestamp:asc). If the Sort property is left blank, then the results will be retrieved in document order.
Supports Expression Language: true
LimitIf set, limits the number of results that will be returned.
Supports Expression Language: true
TargetFlow file content
  • Flow file content
  • Flow file attributes
Indi cates where the results should be placed. In the case of 'Flow file content', the JSON response will be written as the content of the flow file. In the case of 'Flow file attributes', the original flow file (if applicable) will be cloned for each result, and all return fields will be placed in a flow file attribute of the same name, but prefixed by 'es.result.'

Dynamic Properties:

Dynamic Properties allow the user to specify both the name and value of a property.
NameValueDescription
A URL query parameterThe value to set it toAdds the specified property name/value as a query parameter in the Elasticsearch URL used for processing
Supports Expression Language: true

Relationships:

NameDescription
retryA FlowFile is routed to th is relationship if the document cannot be fetched but attempting the operation again may succeed. Note that if the processor has no incoming connections, flow files may still be sent to this relationship based on the processor properties and the results of the fetch operation.
successAll FlowFiles that are read from Elasticsearch are routed to this relationship.
failureAll FlowFiles that cannot be read from Elasticsearch are routed to this relationship. Note that only incoming flow files will be routed to failure.

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
filenameThe filename attribute is set to the document identifier
es.idThe Elasticsearch document identifier
es.indexThe Elasticsearch index containing the document
es.type The Elasticsearch document type
es.result.*If Target is 'Flow file attributes', the JSON attributes of each result will be placed into corresponding attributes with this prefix.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component allows an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.ScrollElasticsearchHttp/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.ScrollElasticsearchHttp/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.ScrollElasticsearchHttp/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-nar/1.4.0/org.apache.nifi.processors.elasticsearch.ScrollElasticsearchHttp/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +ScrollElasticsearchHttp

ScrollElasticsearchHttp

Description:

Scrolls through an Elasticsearch query using the specified connection properties. This processor is intended to be run on the primary node, and is designed for scrolling through huge result sets, as in the case of a reindex. The state must be cleared before another query can be run. Each page of results is returned, wrapped in a JSON object like so: { "hits" : [ <doc1>, <doc2>, <docn> ] }. Note that the full body of each page of documents will be read into memory before being written to a Flow File for transfer.

Tags:

elasticsearch, query, scroll, read, get, http

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Language, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
Elasticsearch URLElasticsearch URL which will be connected to, including scheme (http, e. g.), host, and port. The default port for the REST API is 9200.
Supports Expression Language: true
SSL Context ServiceController Service API:
SSLContextService
Implementations: StandardSSLContextService
StandardRestrictedSSLContextService
The SSL Context Service used to provide client certificate information for TLS/SSL connections. This service only applies if the Elasticsearch endpoint(s) have been secured with TLS/SSL.
UsernameUsername to access the Elasticse arch cluster
Supports Expression Language: true
PasswordPassword to access the Elasticsearch cluster
Sensitive Property: true
Supports Expression Language: true
Connection Timeout5 secsMax wait time for the connection to the Elasticsearch REST API.
Supports Expression Language: true
Response Timeout15 secsMax wait time for a response from the Elasticsearch REST API.
Supports Expression Language: true
QueryThe Lucene-style query to run against ElasticSearch (e.g., genre:blues AND -artist:muddy)
Supports Expression Language: true
Scroll Duration1mThe scroll duration is how long each search context is kept in memory.
Supports Expression Language: true
Page Size20Determines how many documents to return per page during scrolling.
Supports Expression Language: true
IndexThe name of the index to read from. If the property is set to _all, the query will match across all indexes.
Supports Expression Language: true< /td>
TypeThe (optional) type of this query, used by Elasticsearch for indexing and searching. If the property is empty, the the query will match across all types.
Supports Expression Language: true
FieldsA comma-separated list of fields to retrieve from the document. If the Fields property is left blank, then the entire document's source will be retrieved.
Supports Expression Language: true
SortA sort parameter (e.g., timestamp:asc). If the Sort property is left blank, then the results will be retrieved in document order.
Supports Expression Language: true

Dynamic Properties:

Dynamic Properties allow the user to specify both the name and value of a property.
NameValueDescription
A URL query parameterThe value to set it toAdds the specified property name/value as a query parameter in the Elasticsearch URL used for processing
Supports Expression Language: true

Relationships:

NameDescription
successAll FlowFiles that are read from Elasticsearch are routed to this relationship.
failureAll FlowFiles that cannot be read from Elasticsearch are routed to this relationship. Note that only incoming flow files will be routed to failure.

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
es .indexThe Elasticsearch index containing the document
es.typeThe Elasticsearch document type

State management:

ScopeDescription
LOCALAfter each successful scroll page, the latest scroll_id is persisted in scrollId as input for the next scroll call. Once the entire query is complete, finishedQuery state will be set to true, and the processor will not execute unless this is cleared.

Restricted:

This component is not restricted.

Input requirement:

This component does not allow an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumeEWS/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumeEWS/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumeEWS/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumeEWS/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +ConsumeEWS

ConsumeEWS

Description:

Consumes messages from Microsoft Exchange using Exchange Web Services. The raw-bytes of each received email message are written as contents of the FlowFile

Tags:

Email, EWS, Exchange, Get, Ingest, Ingress, Message, Consume

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Lan guage, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
User NameUser Name used for authentication and authorization with Email server.
Supports Expression Language: true
PasswordPassword used for authentication and authorization with Email server.
Sensitive Property: true
Supports Expression Language: true
Fol derINBOXEmail folder to retrieve messages from (e.g., INBOX)
Supports Expression Language: true
Fetch Size10Specify the maximum number of Messages to fetch per call to Email Server.
Supports Expression Language: true
Delete Messagesfalse
  • true
  • false
Specify whether mail messages should be deleted after retrieval.
Connection timeout30 secThe amount of time to wait to connect to Email server
Supports Expression Language: true
Exchange VersionExchange2010_SP2
  • Exchange2007_SP1
  • Exchange2010
  • Exchange2010_SP1
  • Exchange2010_SP2
What version of Exchange Server the server is running.
EWS URLURL of the EWS Endpoint. Required if Autodiscover is false.
Auto Discover URLtrue
  • true
  • false
Whether or not to use the Exchange email address to Autodiscover the EWS endpoint URL.
Mark Messages as Readtrue
  • true
  • false
Specify if messages should be marked as read after retrieval.
Original Headers to IncludeComma delimited list specifying which headers from the original message to include in the exported email message. Blank means copy all headers. Some headers can cause problems with message parsing, specifically the 'Content-Type' header.
Original Headers to ExcludeComma delimited list specifying which headers from the original message to exclude in the exported email message. Blank means don't exclude any headers.

Relationships:

NameDescription
successAll messages that are the are successfully received from Email server and converted to FlowFiles are routed to this relationship

Reads Attributes: < /h3>None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component does not allow an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumeIMAP/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumeIMAP/additionalDetails.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumeIMAP/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumeIMAP/additionalDetails.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1,58 @@ + + + + + +ConsumeIMAP + + + + + +

Description:

+

This Processor consumes email messages via IMAP protocol and sends the content of an email message as content of the Flow File. + Content of the incoming email message is written as raw bytes to the content of the outgoing Flow File. +

+ +

Different email providers may require additional Java Mail properties which could be provided as dynamic properties. + For example, below is a sample configuration for GMail: + +

+

+ Processor's static properties: +

+ Processor's dynamic properties: + +

+

+ Another useful property is mail.debug which allows Java Mail API to print protocol messages to the console helping you to both understand what's going on as well as debug issues. +

+

+ For the full list of available Java Mail properties please refer to here +

+ + Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumeIMAP/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumeIMAP/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumeIMAP/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumeIMAP/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +ConsumeIMAP

ConsumeIMAP

Description:

Consumes messages from Email Server using IMAP protocol. The raw-bytes of each received email message are written as contents of the FlowFile

Additional Details...

Tags:

Email, Imap, Get, Ingest, Ingress, Message, Consume

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Language, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
Host NameNetwork address of Email server (e.g., pop.gmail.com, imap.gmail.com . . .)
Supports Expression Language: true
PortNumeric value identifying Port of Email server (e.g., 993)
Supports Expression Language: true
User NameUser Name used for authentication and authorization with Email server.
Supports Expression Language: true
PasswordPassword used for authentication and authorization with Email server.
Sensitive Property: true
Supports Expression Language: true
FolderINBOXEmail folder to retrieve messages from (e.g., INBOX)
Supports Expression Language: true
Fetch Size10Specify the maximum number of Messages to fetch per call to Email Server.
Supports Expression Language: true
Delete Messagesfalse
  • true
  • false
Specify whether mail messages should be deleted after retrieval.
Connection timeout30 secThe amount of time to wait to connect to Email server
Supports Expression Language: true
Mark Messages as Readfalse
  • true
  • false
Specify if messages should be marked as read after retrieval.
Use SSLtrue
  • true
  • false
Specifie s if IMAP connection must be obtained via SSL encrypted connection (i.e., IMAPS)

Relationships:

NameDescription
successAll messages that are the are successfully received from Email server and converted to FlowFiles are routed to this relationship

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component does not allow an incoming relationship. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumePOP3/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumePOP3/additionalDetails.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumePOP3/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumePOP3/additionalDetails.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1,57 @@ + + + + + +ConsumePOP3 + + + + + +

Description:

+

This Processor consumes email messages via POP3 protocol and sends the content of an email message as content of the Flow File. + Content of the incoming email message is written as raw bytes to the content of the outgoing Flow File. +

+ +

Since different serves may require different Java Mail + properties such properties could be provided via dynamic properties. + For example, below is a sample configuration for GMail: +

+

+ Processor's static properties: +

+ Processor's dynamic properties: + +

+

+ Another useful property is mail.debug which allows Java Mail API to print protocol messages to the console helping you to both understand what's going on as well as debug issues. +

+

+ For the full list of available Java Mail properties please refer to here +

+ + Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumePOP3/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumePOP3/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumePOP3/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ConsumePOP3/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +ConsumePOP3

ConsumePOP3

Description:

Consumes messages from Email Server using POP3 protocol. The raw-bytes of each received email message are written as contents of the FlowFile

Additional Details...

Tags:

Email, POP3, Get, Ingest, Ingress, Message, Consume

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Language, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
Host NameNetwork address of Email server (e.g., pop.gmail.com, imap.gmail.com . . .)
Supports Expression Language: true
PortNumeric value identifying Port of Email server (e.g., 993)
Supports Expression Language: true
User NameUser Name used for authentication and authorization with Email server.
Supports Expression Language: true
PasswordPassword used for authentication and authorization with Email server.
Sensitive Property: true
Supports Expression Language: true
FolderINBOXEmail folder to retrieve messages from (e.g., INBOX)
Supports Expression Language: true
Fetch Size10Specify the maximum number of Messages to fetch per call to Email Server.
Supports Expression Language: true
Delete Messagesfalse
  • true
  • false
Specify whether mail messages should be deleted after retrieval.
Connection timeout30 secThe amount of time to wait to connect to Email server
Supports Expression Language: true

Relationships:

NameDescription
successAll messages that are the are successfully received from Email server and converted to FlowFiles are routed to this relationship

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restri cted:

This component is not restricted.

Input requirement:

This component does not allow an incoming relationship. \ No newline at end of file