PutMongo

From commits-return-42384-archive-asf-public=cust-asf.ponee.io@nifi.apache.org Mon Feb 3 21:53:49 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 160E8180647 for ; Mon, 3 Feb 2020 22:53:48 +0100 (CET) Received: (qmail 78903 invoked by uid 500); 3 Feb 2020 21:53:41 -0000 Mailing-List: contact commits-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@nifi.apache.org Delivered-To: mailing list commits@nifi.apache.org Received: (qmail 78324 invoked by uid 99); 3 Feb 2020 21:53:40 -0000 Received: from Unknown (HELO svn01-us-east.apache.org) (13.90.137.153) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Feb 2020 21:53:39 +0000 Received: from svn01-us-east.apache.org (svn01-us-east.apache.org [127.0.0.1]) by svn01-us-east.apache.org (ASF Mail Server at svn01-us-east.apache.org) with ESMTP id 7ED5D17A15F for ; Mon, 3 Feb 2020 21:53:39 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: svn commit: r1873556 [24/48] - in /nifi/site/trunk/docs/nifi-docs: ./ components/org.apache.nifi/nifi-ambari-nar/1.11.1/ components/org.apache.nifi/nifi-ambari-nar/1.11.1/org.apache.nifi.reporting.ambari.AmbariReportingTask/ components/org.apache.nifi/... Date: Mon, 03 Feb 2020 21:53:33 -0000 To: commits@nifi.apache.org From: joewitt@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20200203215339.7ED5D17A15F@svn01-us-east.apache.org> Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.PutMongo/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.PutMongo/additionalDetails.html?rev=1873556&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.PutMongo/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.PutMongo/additionalDetails.html Mon Feb 3 21:53:27 2020 @@ -0,0 +1,77 @@ + + + + + + PutMongo + + + + + +

Description:

+ This processor is a general purpose processor for inserting, upserting and updating MongoDB collections. +

Inserting Documents

+ Each flowfile is assumed to contain only a single MongoDB document to be inserted. The contents must be valid JSON. + The input the Mongo shell accepts should not be confused with valid JSON. It does not support batch writes at this time. +

Updating and Upserting

+ +

Update Modes

+ There are two methods for choosing what gets written to a document when updating: +

Whole document - the entire document is replaced with the contents of the flowfile.
With Operators Enabled - the document in the flowfile content will be assumed to have update operators such as + $set and will be used to update particular fields. The whole document will not be replaced.

+ +

+ There are two ways to update: +

Update Key - use one or more keys from the document.
Update Query - use a totally separate query that is not derived from the document.

+ +

Update Key

+ The update key method takes keys from the document and builds a query from them. It will attempt to parse the _id + field as an ObjectID type if that is one of the keys that is specified in the configuration field. Multiple keys + can be specified by separating them with commas. This configuration field supports Expression Language, so it can + derived in part or entirely from flowfile attributes. +

+ +

Update Query

+ The update query method takes a valid JSON document as its value and uses it to find one or more documents to update. + This field supports Expression Language, so it can be derived in part or entirely from flowfile attributes. It is possible, + for instance, to put an attribute named update_query on a flowfile and specify ${update_query} in the + configuration field so it will load the value from the flowfile. +

+ +

Upserts

+ If the upsert mode is enabled, PutMongo will insert a new document that matches the search criteria (be it a user-supplied + query or one built from update keys) and give it the properties that are specified in the JSON document provided in the + flowfile content. This feature should be used carefully, as it can result in incomplete data being added to MongoDB. +

+ + \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.PutMongo/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.PutMongo/index.html?rev=1873556&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.PutMongo/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.PutMongo/index.html Mon Feb 3 21:53:27 2020 @@ -0,0 +1 @@ +PutMongo

Description:

Writes the contents of a FlowFile to MongoDB

Additional Details...

Tags:

mongodb, insert, update, write, put

Properties:

Name	Default Value	Allowable Values	Description
Client Service		Controller Service API: MongoDBClientService Implementation: MongoDBControllerService	If configured, this property will use the assigned client service for connection pooling.
Mongo URI			MongoURI, typically of the form: mongodb://host1[:port1][,host2[:port2],...] Supports Expression Language: true (will be evaluated using variable registry only)
Mongo Database Name			The name of the database to use Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Mongo Collection Name			The name of the collection to use Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
SSL Context Service		Controller Service API: SSLContextService Implementations: StandardSSLContextService StandardRestrictedSSLContextService	The SSL Context Service used to provide client certificate information for TLS/SSL connections.
Client Auth	REQUIRED	WANT REQUIRED NONE	Client authentication policy when connecting to secure (TLS/SSL) cluster. Possible values are REQUIRED, WANT, NONE. This property is only used when an SSL Context has been defined and enabled.
Mode	insert	insert update	Indicates whether the processor should insert or update content
Upsert	false	true false	When true, inserts a document if no document matches the update query criteria; this property is valid only when using update mode, otherw ise it is ignored
Update Query Key			Key name used to build the update query criteria; this property is valid only when using update mode, otherwise it is ignored. Example: _id Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Update Query			Specify a full MongoDB query to be used for the lookup query to do an update/upsert. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Update Mode	doc	With whole document With operators enabled	Choose an update mode. You can either sup ply a JSON document to use as a direct replacement or specify a document that contains update operators like $set, $unset, and $inc. When Operators mode is enabled, the flowfile content is expected to be the operator part for example: {$set:{"key": "value"},$inc:{"count":1234}} and the update query will come from the configured Update Query property.
Write Concern	ACKNOWLEDGED	ACKNOWLEDGED UNACKNOWLEDGED FSYNCED JOURNALED REPLICA_ACKNOWLEDGED MAJORITY	The write concern to use
Character Set	UTF-8		The Character Set in which the data is encoded

Relationships:

Name	Description
success	All F lowFiles that are written to MongoDB are routed to this relationship
failure	All FlowFiles that cannot be written to MongoDB are routed to this relationship

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

Resource	Description
MEMORY	An instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.

\ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.PutMongoRecord/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.PutMongoRecord/index.html?rev=1873556&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.PutMongoRecord/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.PutMongoRecord/index.html Mon Feb 3 21:53:27 2020 @@ -0,0 +1 @@ +PutMongoRecord

Description:

This processor is a record-aware processor for inserting data into MongoDB. It uses a configured record reader and schema to read an incoming record set from the body of a flowfile and then inserts batches of those records into a configured MongoDB collection. This processor does not support updates, deletes or upserts. The number of documents to insert at a time is controlled by the "Insert Batch Size" configuration property. This value should be set to a reasonable size to ensure that MongoDB is not overloaded with too many inserts at once.

Tag s:

mongodb, insert, record, put

Properties:

Name	Default Value	Allowable Values	Description
Client Service		Controller Service API: MongoDBClientService Implementation: MongoDBControllerService	If configured, this property will use the assigned client service for connection pooling.
Mongo URI			MongoURI, typically of the form: mongodb://host1[:port1][,host2[:port2],...] Supports Expression Language: true (will be evaluated using variable registry only)
Mongo Database Name			The name of the database to use Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Mongo Collection Name			The name of the collection to use Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
SSL Context Service		Controller Ser vice API: SSLContextService Implementations: StandardSSLContextService StandardRestrictedSSLContextService	The SSL Context Service used to provide client certificate information for TLS/SSL connections.
Client Auth	REQUIRED	WANT REQUIRED NONE	Client authentication policy when connecting to secure (TLS/SSL) cluster. Possible values are REQUIRED, WANT, NONE. This property is only used when an SSL Context has been defined and enabled.
Write Concern	ACKNOWLEDGED	ACKNOWLEDGED UNACKNOWLEDGED FSYNCED JOURNALED REPLICA_ACKNOWLEDGED MAJORITY	The write concern to use
Record Reader		Controller Service API: RecordReaderFactory Implementations: ScriptedReader JsonTreeReader ParquetReader JsonPathReader GrokReader Syslog5424Reader XMLReader AvroReader CSVReader SyslogReader	Specifies the Controller Service to use for parsing incoming data and determining the data's schema
Insert Batch Size	100		The number of record s to group together for one single insert operation against MongoDB.

Relationships:

Name	Description
success	All FlowFiles that are written to MongoDB are routed to this relationship
failure	All FlowFiles that cannot be written to MongoDB are routed to this relationship

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

None specified. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.RunMongoAggregation/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.RunMongoAggregation/additionalDetails.html?rev=1873556&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.RunMongoAggregation/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.RunMongoAggregation/additionalDetails.html Mon Feb 3 21:53:27 2020 @@ -0,0 +1,44 @@ + + + + + + RunMongoAggregation + + + + + +

Description:

+ This processor runs a MongoDB aggregation query based on user-defined settings. The + following is an example of such a query (and what the expected input looks like): +

+[{
+   "$project": {
+        "domain": 1
+    },
+    "$group": {
+         "_id": { "domain": "$domain" },
+         "total": {
+             "$sum": 1
+         }
+     }
+}]
+

+ + \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.RunMongoAggregation/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.RunMongoAggregation/index.html?rev=1873556&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.RunMongoAggregation/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.RunMongoAggregation/index.html Mon Feb 3 21:53:27 2020 @@ -0,0 +1 @@ +RunMongoAggregation

Description:

A processor that runs an aggregation query whenever a flowfile is received.

Additional Details...

Tags:

mongo, aggregation, aggregate

Properties:

Name	Default Value	Allowable Values	Description
Client Service		Controller Service API: MongoDBClientService Implementation: MongoDBControllerService	If configured, this property will use the assigned client service for connection pooling.
Mongo URI			MongoURI, typically of the form: mongodb://host1[:port1][,host2[:port2],...] Supports Expression Language: true (will be evaluated using variable registry only)
Mongo Database Name			The name of the database to use Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Mongo Collection Name			The name of the collection to use Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
SSL Context Service		Controller Service API: SSLContextService Implementations: StandardSSLContextService StandardRestrictedSSLContextService	The SSL Context Service used to provide client certificate information for TLS/SSL connections.
Client Auth	REQUIRED	WANT REQUIRED NONE	Client authentication policy when connecting to secure (TLS/SSL) cluster. Possible values are REQUIRED, WANT, NONE. This property is only used when an SSL Context has been defined and enabled.
Character Set	UTF-8		Specifies the character set of the document data. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Query			The aggregation query to be executed. Sup ports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Allow Disk Use	false	true false	Set this to true to enable writing data to temporary files to prevent exceeding the maximum memory use limit during aggregation pipeline staged when handling large datasets.
JSON Type	Extended	Extended JSON Standard JSON	By default, MongoDB's Java driver returns "extended JSON". Some of the features of this variant of JSON may cause problems for other JSON parsers that expect only standard JSON types and conventions. This configuration setting controls whether to use extended JSON or provide a clean view that conforms to standard JSON.
Query Output Attribute			If set, the query will be written to a specified attribute on the output flowfiles. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Batch Size	100		The number of elements returned from the server in one batch.
Results Per FlowFile	1		How many results to put into a flowfile at once. The whole body will be treated as a JSON array of results.
Date Format	yyyy-MM-dd'T'HH:mm:ss'Z'		The date format string to use for formatting Date fields that are returned from Mongo. It is only applied when the JSON output format is set to Standard JSON. Full documentation for format characters can be found here: https://docs.oracle.com/javase/8/docs/api/java/text/SimpleDateFormat.html Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
SSL Context Service		Controller Service API: SSLContextService Implementations: StandardSSLContextService StandardRestrictedSSLContextService	The SSL Context Service used to provide client certificate information for TLS/SSL connections.
Client Auth	REQUIRED	WANT REQUIRED NONE	Client authentication policy when connecting to secure (TLS/SSL) cluster. Possible values are REQUIRED, WANT, NONE. This property is only used when an SSL Context has been defined and enabled.

Relationships:

NameDescriptionresultsThe result set of the aggregation will be sent to this relationship.failureThe input flowfile gets sent to this relationship when the query fails.originalThe input flowfile gets sent to this relationship when the query succeeds.

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component allows an incoming relationship.

System Resource Considerations:

None specified. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.DeleteGridFS/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.DeleteGridFS/additionalDetails.html?rev=1873556&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.DeleteGridFS/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.DeleteGridFS/additionalDetails.html Mon Feb 3 21:53:27 2020 @@ -0,0 +1,32 @@ + + + + + + DeleteGridFS + + + + + +

Description:

+ This processor retrieves one or more files from GridFS. The query to execute can be either provided in the query + configuration parameter or generated from the value pulled from the filename configuration parameter. Upon successful + execution, it will append the query that was executed as an attribute on the flowfile that was processed. +

+ + \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.DeleteGridFS/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.DeleteGridFS/index.html?rev=1873556&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.DeleteGridFS/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.DeleteGridFS/index.html Mon Feb 3 21:53:27 2020 @@ -0,0 +1 @@ +DeleteGridFS

Description:

Deletes a file from GridFS using a file name or a query.

Additional Details...

Tags:

gridfs, delete, mongodb

Properties:

Name	Allowable Values	Description
Client Service	Controller Service API: MongoDBClientService Implementation: MongoDBControllerService	The MongoDB client service to use for database connections.
Mongo Database Name		The name of the database to use Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Bucket Name		The GridFS bucket where the files will be stored. If left blank, it will use the default value 'fs' that the MongoDB client driver uses. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
File Name		The name of the file in the bucket that is the target of this processor. GridFS file names do not include path information because GridFS does not sort files into folders within a bucket. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Query		A valid MongoDB query to use to find and delete one or more files from GridFS. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Query Output Attribut e		If set, the query will be written to a specified attribute on the output flowfiles. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)

Relationships:

Name	Description
success	When the operation succeeds, the flowfile is sent to this relationship.
failure	When there is a failure processing the flowfile, it goes to this relationship.

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

None specified. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.FetchGridFS/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.FetchGridFS/additionalDetails.html?rev=1873556&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.FetchGridFS/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.FetchGridFS/additionalDetails.html Mon Feb 3 21:53:27 2020 @@ -0,0 +1,43 @@ + + + + + + FetchGridFS + + + + + +

Description:

+ This processor retrieves one or more files from GridFS. The query can be provided in one of three ways: +

+ +

Query configuration parameter.
Built for you by configuring the filename parameter. (Note: this is just a filename, Mongo queries cannot be + embedded in the field).
Retrieving the query from the flowfile contents.

+ +

+ The processor can also be configured to either commit only once at the end of a fetch operation or after each file + that is retrieved. Multiple commits is generally only necessary when retrieving a lot of data from GridFS as measured + in total data size, not file count, to ensure that the disks NiFi is using are not overloaded. +

+ + \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.FetchGridFS/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.FetchGridFS/index.html?rev=1873556&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.FetchGridFS/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.FetchGridFS/index.html Mon Feb 3 21:53:27 2020 @@ -0,0 +1 @@ +FetchGridFS

Description:

Retrieves one or more files from a GridFS bucket by file name or by a user-defined query.

Additional Details...

Tags:

fetch, gridfs, mongo

Properties:

Name	Default Value	Allowable Values	Description
Client Service		Controller Service API: MongoDBClientService Implementation: MongoDBControllerService	The MongoDB client service to use for database connections.
Mongo Database Name			The name of the database to use Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Bucket Name			The GridFS bucket where the files will b e stored. If left blank, it will use the default value 'fs' that the MongoDB client driver uses. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
File Name			The name of the file in the bucket that is the target of this processor. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Query			A valid MongoDB query to use to fetch one or more files from GridFS. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Query Output Attribute			If set, the query will be written to a specified attribute on the output flowfiles. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Operation Mode	all-at-once	Full Query Fetch Stream Query Results	This option controls when results are made available to downstream processors. If St ream Query Results is enabled, provenance will not be tracked relative to the input flowfile if an input flowfile is received and starts the query. In Stream Query Results mode errors will be handled by sending a new flowfile with the original content and attributes of the input flowfile to the failure relationship. Streaming should only be used if there is reliable connectivity between MongoDB and NiFi.

Relationships:

Name	Description
success	When the operation succeeds, the flowfile is sent to this relationship.
failure	When there is a failure processing the flowfile, it goes to this relationship.
original	The original input flowfile goes to this relationship if the query does not cause an error

Reads Attributes:

None specified.

Writes Attributes:

Name	Descripti on
gridfs.file.metadata	The custom metadata stored with a file is attached to this property if it exists.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

None specified. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.PutGridFS/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.PutGridFS/additionalDetails.html?rev=1873556&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.PutGridFS/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.PutGridFS/additionalDetails.html Mon Feb 3 21:53:27 2020 @@ -0,0 +1,58 @@ + + + + + + PutGridFS + + + + + +

Description:

+ This processor puts a file with one or more user-defined metadata values into GridFS in the configured bucket. It + allows the user to define how big each file chunk will be during ingestion and provides some ability to intelligently + attempt to enforce file uniqueness using filename or hash values instead of just relying on a database index. +

GridFS File Attributes

+ PutGridFS allows for flowfile attributes that start with a configured prefix to be added to the GridFS + document. These can be very useful later when working with GridFS for providing metadata about a file. +

Chunk Size

+ GridFS splits up file into chunks within Mongo documents as the file is ingested into the database. The chunk size + configuration parameter configures the maximum size of each chunk. This field should be left at its default value + unless there is a specific business case to increase or decrease it. +

Uniqueness Enforcement

+ There are four operating modes: +

No enforcement at the application level.
Enforce by unique file name.
Enforce by unique hash value.
Use both hash and file name.

+ The hash value by default is taken from the attribute hash.value which can be generated by configuring a + HashContent processor upstream of PutGridFS. Both this and the name option use a query on the existing + data to see if a file matching that criteria exists before attempting to write the flowfile contents. +

+ + \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.PutGridFS/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.PutGridFS/index.html?rev=1873556&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.PutGridFS/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.11.1/org.apache.nifi.processors.mongodb.gridfs.PutGridFS/index.html Mon Feb 3 21:53:27 2020 @@ -0,0 +1 @@ +PutGridFS

Description:

Writes a file to a GridFS bucket.

Additional Details...

Tags:

mongo, gridfs, put, file, store

Properties:

Name	Default Value	Allo wable Values	Description
Client Service		Controller Service API: MongoDBClientService Implementation: MongoDBControllerService	The MongoDB client service to use for database connections.
Mongo Database Name			The name of the database to use Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Bucket Name			The GridFS bucket where the files will be stored. If left blank, it will use the default value 'fs' that the MongoDB client driver uses. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
File Name			The name of the file in the bucket that is the target of this processor. GridFS file names do not include path information because GridFS does not sort files into folders within a bucket. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
File Properties Prefix			Attributes that have this prefix will be added to the file stored in GridFS as metadata. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Enforce Uniqueness	none	None Both Name Hash	When enabled, this option will ensure that uniqueness is enforced on the bucket. It will do so by creating a MongoDB index that matches your selection. It should ideally be configured once when the bucket is created for the first time because it could take a long time to build on an existing bucket wit a lot of data.
Hash Attribute	hash.value		If uniquness enforcement is enabled and the file hash is part of the constraint, this must be set to an attribute that exists on all incoming flowfiles. Supports Expression Language: true (will be evaluated using variable registry only)
Chunk Size	256 KB		Controls the maximum size of each chunk of a file uploaded into GridFS. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)

Relationships:

Name	Description
success	When the operation succeeds, the flowfile i s sent to this relationship.
duplicate	Flowfiles that fail the duplicate check are sent to this relationship.
failure	When there is a failure processing the flowfile, it goes to this relationship.

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

None specified. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-services-nar/1.11.1/org.apache.nifi.mongodb.MongoDBControllerService/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-services-nar/1.11.1/org.apache.nifi.mongodb.MongoDBControllerService/index.html?rev=1873556&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-services-nar/1.11.1/org.apache.nifi.mongodb.MongoDBControllerService/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-services-nar/1.11.1/org.apache.nifi.mongodb.MongoDBControllerService/index.html Mon Feb 3 21:53:27 2020 @@ -0,0 +1 @@ +MongoDBControllerService

Description:

Provides a controller service that configures a connection to MongoDB and provides access to that connection to other Mongo-related components.

Tags:

mongo, mongodb, service

Properties:

NameDefault ValueAllowable ValuesDescriptionMongo URIMongoURI, typically of the form: mongodb://host1[:port1][,host2[:port2],...]
Supports Expression Language: true (will be evaluated using variable registry only)SSL Context ServiceController Service API:
SSLContextService
Implementations: StandardSSLContextService
StandardRestrictedSSLContextServiceThe SSL Context Service used to provi de client certificate information for TLS/SSL connections.Client AuthREQUIRED

WANT
REQUIRED
NONE

Client authentication policy when connecting to secure (TLS/SSL) cluster. Possible values are REQUIRED, WANT, NONE. This property is only used when an SSL Context has been defined and enabled.

State management:

This component does not store state.

Restricted:

This component is not restricted.

System Resource Considerations:

None specified. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-services-nar/1.11.1/org.apache.nifi.mongodb.MongoDBLookupService/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-services-nar/1.11.1/org.apache.nifi.mongodb.MongoDBLookupService/index.html?rev=1873556&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-services-nar/1.11.1/org.apache.nifi.mongodb.MongoDBLookupService/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-services-nar/1.11.1/org.apache.nifi.mongodb.MongoDBLookupService/index.html Mon Feb 3 21:53:27 2020 @@ -0,0 +1,4 @@ +MongoDBLookupService

Description:

Provides a lookup service based around MongoDB. Each key that is specified +will be added to a query as-is. For example, if you specify the two keys, +user and email, the resulting query will be { "user": "tester", "email": "tester@test.com" }. +The query is limited to the first result (findOne in the Mongo documentation). If no "Lookup Value Field" is specified then the entire MongoDB result document minus the _id field will be returned as a record.

Tags:

mongo, mongodb, lookup, record

Properties:

Name	Default Value	Allowable Values	Description
Schema Access Strategy	infer	Use 'Schema Name' Property Use 'Schema Text' Property Infer from Result	Specifies how to obtain the schema that is to be used for interpreting the data.
Schema Registry		Controller Service API: SchemaRegistry Implementations: ConfluentSchemaRegistry HortonworksSchemaRegistry AvroSchemaRegistry	Specifies the Controller Service to use for the Schema Registry
Schema Name	${schema.name}		Specifies the name of the schema to lookup in the Schema Registry property Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Schema Version			Specifies the version of the schema to lookup in the Schema Registry. If not specified then the latest version of the schema will be retrieved. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Schema Branch			Specifies the name of the branch to use when looking up the schema in the Schema Registry property. If the chosen Schema Registry does not support branching, this value will be ignored. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Schema Text	${avro.schema}		The text of an Avro-formatted Schema Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Client Service		Controller Service API: MongoDBClientService Implementation: MongoDBControllerService	A MongoDB controller service to use with this lookup service.
Mongo Database Name			The name of the database to use Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Mongo Collection Name			The name of the collection to use Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Lookup Value Field			The field whose value will be returned when the lookup key(s) match a record. If not specified then the entire MongoDB result document minus the _id field will be returned as a record.
Projection			Specifies a projection for limiting which fields will be returned.

State management:

This component does not store state.

Restricted:

This component is not restricted.

System Resource Considerations:

None spec ified. \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mqtt-nar/1.11.1/org.apache.nifi.processors.mqtt.ConsumeMQTT/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mqtt-nar/1.11.1/org.apache.nifi.processors.mqtt.ConsumeMQTT/index.html?rev=1873556&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mqtt-nar/1.11.1/org.apache.nifi.processors.mqtt.ConsumeMQTT/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-mqtt-nar/1.11.1/org.apache.nifi.processors.mqtt.ConsumeMQTT/index.html Mon Feb 3 21:53:27 2020 @@ -0,0 +1 @@ +ConsumeMQTT

Description:

Subscribes to a topic and receives messages from an MQTT broker

Tags:

subscribe, MQTT, IOT, consume, listen

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

Name	Default Value	Allowable Values	Description
Broker URI			The URI to use to connect to the MQTT broker (e.g. tcp://localhost:1883). The 'tcp', 'ssl', 'ws' and 'wss' schemes are supported. In order to use 'ssl', the SSL Context Service property must be set.
Client ID			MQTT client ID to use
Username			Username to use when connecting to the broker
Password			Password to use when connecting to the broker Sensit ive Property: true
SSL Context Service		Controller Service API: SSLContextService Implementations: StandardSSLContextService StandardRestrictedSSLContextService	The SSL Context Service used to provide client certificate information for TLS/SSL connections.
Last Will Topic			The topic to send the client's Last Will to. If the Last Will topic and message are not set then a Last Will will not be sent.
Last Will Message		The message to send as the client's Last Will. If the Last Will topic and message are not set then a Last Will will not be sent.
Last Will Retain		true false	Whether to retain the client's Last Will. If the Last Will topic and message are not set then a Last Will will not be sent.
Last Will QoS Level		0 - At most once 1 - At least once 2 - Exactly once	QoS level to be used when publishing the Last Will Message
Session state	true	Clean Session Resume Session	Whether to start afresh or resume previous flows. See the allowable value descriptions for more details.
MQTT Specification Version	0	AUTO v3.1.1 v3.1.0	The MQTT specification version when connecting with the broker. See the allowable value descriptions for more details.
Connection Timeout (seconds)	30		Maximum time interval the client will wait for the network connection to the MQTT server to be established. The default timeout is 30 seconds. A value of 0 disables timeout processing meaning the client will wait until the network connection is made successfully or fails.
Keep Alive Interval (seconds)	60		Defines the maximum time interval between messages sent or received. It enables the client to detect if the server is no longer available, without having to wait for the TCP/IP timeout. The client will ensure that at least one message travels across the network within each keep alive period. In the absence of a data-related message during the time period, the client sends a very small "ping" message, which the server will acknowledge. A value of 0 disables keepalive processing in the client.
Topic Filter			The MQTT topic filter to designate the topics to subscribe to.
Quality of Service(QoS)	0	0 - At most once 1 - At least once 2 - Exactly once	The Quality of Service(QoS) to receive the message with. Accepts values '0', '1' or '2'; '0' for 'at most once', '1' for 'at least once', '2' for 'exactly once'.
Max Queue Size			The MQTT messages are always being sent to subscribers on a topic. If the 'Run Schedule' is significantly behind the rate at which the messages are arriving to this processor then a back up can occur. This property specifies the maximum number of messages this processor will hold in memory at one time.

Relationships:

Name	Description
Message	The MQTT message output

Reads Attributes: None specified.

Writes Attributes:

Name	Description
mqtt.broker	MQTT broker that was the message source
mqtt.topic	MQTT topic on which message was received
mqtt.qos	The quality of service for this message.
mqtt.isDuplicate	Whether or not this message might be a duplicate of one which has already been received.
mqtt.isRetained	Whether or not this message was from a current publisher, or was "retained" by the server as the last message published on the topic.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component does not allow an incoming relationship.

System Resource Considerations:

None specified.