{ "bundles": [ { "@type": "NXBundle", "artifactId": "nuxeo-core-bulk", "artifactVersion": "2023.9.10", "bundleGroup": { "@type": "NXBundleGroup", "bundleIds": [ "org.nuxeo.ecm.core", "org.nuxeo.ecm.core.api", "org.nuxeo.ecm.core.bulk", "org.nuxeo.ecm.core.cache", "org.nuxeo.ecm.core.event", "org.nuxeo.ecm.core.io", "org.nuxeo.ecm.core.mimetype", "org.nuxeo.ecm.core.mongodb", "org.nuxeo.ecm.core.persistence", "org.nuxeo.ecm.core.query", "org.nuxeo.ecm.core.schema", "org.nuxeo.ecm.platform.el" ], "hierarchyPath": "/grp:org.nuxeo.ecm.core", "id": "grp:org.nuxeo.ecm.core", "name": "org.nuxeo.ecm.core", "parentIds": [], "readmes": [], "version": "2023.9" }, "bundleId": "org.nuxeo.ecm.core.bulk", "components": [ { "@type": "NXComponent", "componentClass": "org.nuxeo.ecm.core.scroll.ScrollComponent", "documentation": "\n The scroll service offers an unified API to scroll large result set.\n \n", "documentationHtml": "

\nThe scroll service offers an unified API to scroll large result set.\n

", "extensionPoints": [ { "@type": "NXExtensionPoint", "componentId": "org.nuxeo.ecm.core.scroll.service", "descriptors": [ "org.nuxeo.ecm.core.scroll.ScrollDescriptor" ], "documentationHtml": "", "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.scroll.service/ExtensionPoints/org.nuxeo.ecm.core.scroll.service--scroll", "id": "org.nuxeo.ecm.core.scroll.service--scroll", "label": "scroll (org.nuxeo.ecm.core.scroll.service)", "name": "scroll", "version": "2023.9.10" } ], "extensions": [], "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.scroll.service", "name": "org.nuxeo.ecm.core.scroll.service", "requirements": [], "resolutionOrder": 128, "services": [ { "@type": "NXService", "componentId": "org.nuxeo.ecm.core.scroll.service", "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.scroll.service/Services/org.nuxeo.ecm.core.api.scroll.ScrollService", "id": "org.nuxeo.ecm.core.api.scroll.ScrollService", "overriden": false, "version": "2023.9.10" } ], "startOrder": 816, "version": "2023.9.10", "xmlFileContent": "\n\n\n \n \n \n\n \n\n \n The scroll service offers an unified API to scroll large result set.\n \n\n \n \n \n\n\n\n", "xmlFileName": "/OSGI-INF/scroll-service.xml", "xmlPureComponent": false }, { "@type": "NXComponent", "documentationHtml": "", "extensionPoints": [], "extensions": [ { "@type": "NXContribution", "documentationHtml": "", "extensionPoint": "org.nuxeo.ecm.core.scroll.service--scroll", "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.scroll.contrib.default/Contributions/org.nuxeo.ecm.core.scroll.contrib.default--scroll", "id": "org.nuxeo.ecm.core.scroll.contrib.default--scroll", "registrationOrder": 1, "targetComponentName": { "rawName": "service:org.nuxeo.ecm.core.scroll.service", "name": "org.nuxeo.ecm.core.scroll.service", "type": "service" }, "version": "2023.9.10", "xml": "\n \n \n \n " } ], "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.scroll.contrib.default", "name": "org.nuxeo.ecm.core.scroll.contrib.default", "requirements": [ "org.nuxeo.ecm.core.scroll.service" ], "resolutionOrder": 130, "services": [], "startOrder": 156, "version": "2023.9.10", "xmlFileContent": "\n\n org.nuxeo.ecm.core.scroll.service\n \n \n \n \n \n\n", "xmlFileName": "/OSGI-INF/scroll-contrib.xml", "xmlPureComponent": true }, { "@type": "NXComponent", "componentClass": "org.nuxeo.ecm.core.bulk.BulkComponent", "declaredStartOrder": -600, "documentation": "\n The Bulk Service allows to execute a Bulk Command. This consists of two steps:\n
    \n
  • Creation of a document set by scrolling the database
    • \n
  • Execution of a bulk on document set
    • \n
\n", "documentationHtml": "

\nThe Bulk Service allows to execute a Bulk Command. This consists of two steps:\n

  • Creation of a document set by scrolling the database
  • Execution of a bulk on document set
", "extensionPoints": [ { "@type": "NXExtensionPoint", "componentId": "org.nuxeo.ecm.core.bulk", "descriptors": [ "org.nuxeo.ecm.core.bulk.BulkActionDescriptor" ], "documentation": "\n Allows to define the bulk actions.\n\n A bulk action is a stream processing composed of one or more computations.\n\n A bulk action can be made available through http APIs, default is false.\n\n The bucket size determines the number of document ids per record at the scroller level, default is 100.\n\n The batch size determines the number of document ids handled per transactions at the computation level, default is 25.\n\n \n \n \n \n \n\n", "documentationHtml": "

\nAllows to define the bulk actions.\n

\nA bulk action is a stream processing composed of one or more computations.\n

\nA bulk action can be made available through http APIs, default is false.\n

\nThe bucket size determines the number of document ids per record at the scroller level, default is 100.\n

\nThe batch size determines the number of document ids handled per transactions at the computation level, default is 25.\n

\n

    <extension point="actions" target="org.nuxeo.ecm.core.bulk">\n        <action batchSize="25" bucketSize="100" name="internalAction"/>\n        <action batchSize="25" bucketSize="100" httpEnabled="true" name="publicAction"/>\n    </extension>\n

", "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.bulk/ExtensionPoints/org.nuxeo.ecm.core.bulk--actions", "id": "org.nuxeo.ecm.core.bulk--actions", "label": "actions (org.nuxeo.ecm.core.bulk)", "name": "actions", "version": "2023.9.10" } ], "extensions": [], "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.bulk", "name": "org.nuxeo.ecm.core.bulk", "requirements": [], "resolutionOrder": 131, "services": [ { "@type": "NXService", "componentId": "org.nuxeo.ecm.core.bulk", "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.bulk/Services/org.nuxeo.ecm.core.bulk.BulkService", "id": "org.nuxeo.ecm.core.bulk.BulkService", "overriden": false, "version": "2023.9.10" }, { "@type": "NXService", "componentId": "org.nuxeo.ecm.core.bulk", "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.bulk/Services/org.nuxeo.ecm.core.bulk.BulkAdminService", "id": "org.nuxeo.ecm.core.bulk.BulkAdminService", "overriden": false, "version": "2023.9.10" } ], "startOrder": 4, "version": "2023.9.10", "xmlFileContent": "\n\n\n \n The Bulk Service allows to execute a Bulk Command. This consists of two steps:\n
    \n
  • Creation of a document set by scrolling the database
    • \n
  • Execution of a bulk on document set
    • \n
\n \n\n \n\n \n \n \n \n\n \n \n Allows to define the bulk actions.\n\n A bulk action is a stream processing composed of one or more computations.\n\n A bulk action can be made available through http APIs, default is false.\n\n The bucket size determines the number of document ids per record at the scroller level, default is 100.\n\n The batch size determines the number of document ids handled per transactions at the computation level, default is 25.\n\n \n \n \n \n \n \n \n \n \n\n\n", "xmlFileName": "/OSGI-INF/bulk-component.xml", "xmlPureComponent": false }, { "@type": "NXComponent", "documentationHtml": "", "extensionPoints": [], "extensions": [ { "@type": "NXContribution", "documentationHtml": "", "extensionPoint": "org.nuxeo.ecm.core.io.MarshallerRegistry--marshallers", "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.bulk.io/Contributions/org.nuxeo.ecm.core.bulk.io--marshallers", "id": "org.nuxeo.ecm.core.bulk.io--marshallers", "registrationOrder": 2, "targetComponentName": { "rawName": "service:org.nuxeo.ecm.core.io.MarshallerRegistry", "name": "org.nuxeo.ecm.core.io.MarshallerRegistry", "type": "service" }, "version": "2023.9.10", "xml": "\n \n \n \n \n " } ], "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.bulk.io", "name": "org.nuxeo.ecm.core.bulk.io", "requirements": [], "resolutionOrder": 135, "services": [], "startOrder": 128, "version": "2023.9.10", "xmlFileContent": "\n\n\n \n \n \n \n \n \n\n\n", "xmlFileName": "/OSGI-INF/bulk-io-contrib.xml", "xmlPureComponent": true }, { "@type": "NXComponent", "documentationHtml": "", "extensionPoints": [], "extensions": [ { "@type": "NXContribution", "documentationHtml": "", "extensionPoint": "org.nuxeo.runtime.avro--schema", "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.bulk.config/Contributions/org.nuxeo.ecm.core.bulk.config--schema", "id": "org.nuxeo.ecm.core.bulk.config--schema", "registrationOrder": 0, "targetComponentName": { "rawName": "service:org.nuxeo.runtime.avro", "name": "org.nuxeo.runtime.avro", "type": "service" }, "version": "2023.9.10", "xml": "\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n " }, { "@type": "NXContribution", "documentationHtml": "", "extensionPoint": "org.nuxeo.runtime.stream.service--logConfig", "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.bulk.config/Contributions/org.nuxeo.ecm.core.bulk.config--logConfig", "id": "org.nuxeo.ecm.core.bulk.config--logConfig", "registrationOrder": 0, "targetComponentName": { "rawName": "service:org.nuxeo.runtime.stream.service", "name": "org.nuxeo.runtime.stream.service", "type": "service" }, "version": "2023.9.10", "xml": "\n \n \n \n \n \n \n \n \n " }, { "@type": "NXContribution", "documentationHtml": "", "extensionPoint": "org.nuxeo.runtime.ConfigurationService--configuration", "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.bulk.config/Contributions/org.nuxeo.ecm.core.bulk.config--configuration", "id": "org.nuxeo.ecm.core.bulk.config--configuration", "registrationOrder": 65, "targetComponentName": { "rawName": "service:org.nuxeo.runtime.ConfigurationService", "name": "org.nuxeo.runtime.ConfigurationService", "type": "service" }, "version": "2023.9.10", "xml": "\n 200\n 300\n 2d\n false\n " }, { "@type": "NXContribution", "documentationHtml": "", "extensionPoint": "org.nuxeo.runtime.stream.service--streamProcessor", "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.bulk.config/Contributions/org.nuxeo.ecm.core.bulk.config--streamProcessor", "id": "org.nuxeo.ecm.core.bulk.config--streamProcessor", "registrationOrder": 15, "targetComponentName": { "rawName": "service:org.nuxeo.runtime.stream.service", "name": "org.nuxeo.runtime.stream.service", "type": "service" }, "version": "2023.9.10", "xml": "\n \n \n \n \n \n \n \n \n \n\n \n\n " }, { "@type": "NXContribution", "documentationHtml": "", "extensionPoint": "org.nuxeo.ecm.core.bulk--actions", "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.bulk.config/Contributions/org.nuxeo.ecm.core.bulk.config--actions", "id": "org.nuxeo.ecm.core.bulk.config--actions", "registrationOrder": 12, "targetComponentName": { "rawName": "service:org.nuxeo.ecm.core.bulk", "name": "org.nuxeo.ecm.core.bulk", "type": "service" }, "version": "2023.9.10", "xml": "\n \n \n \n \n \n \n " }, { "@type": "NXContribution", "documentationHtml": "", "extensionPoint": "org.nuxeo.runtime.stream.service--streamProcessor", "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.bulk.config/Contributions/org.nuxeo.ecm.core.bulk.config--streamProcessor1", "id": "org.nuxeo.ecm.core.bulk.config--streamProcessor1", "registrationOrder": 16, "targetComponentName": { "rawName": "service:org.nuxeo.runtime.stream.service", "name": "org.nuxeo.runtime.stream.service", "type": "service" }, "version": "2023.9.10", "xml": "\n \n \n \n \n\n \n \n \n \n\n \n \n \n \n\n \n \n \n \n\n " } ], "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk/org.nuxeo.ecm.core.bulk.config", "name": "org.nuxeo.ecm.core.bulk.config", "requirements": [ "org.nuxeo.runtime.stream.service" ], "resolutionOrder": 824, "services": [], "startOrder": 127, "version": "2023.9.10", "xmlFileContent": "\n\n\n org.nuxeo.runtime.stream.service\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n \n \n\n \n \n \n \n \n \n \n \n \n \n \n\n \n \n 200\n ${nuxeo.core.bulk.scroller.scroll.keepAliveSeconds:=300}\n 2d\n false\n \n\n \n \n \n \n \n \n \n \n \n \n \n\n \n\n \n\n \n \n\n \n \n \n \n \n \n \n \n\n \n \n \n \n \n \n\n \n \n \n \n\n \n \n \n \n\n \n \n \n \n\n \n\n\n", "xmlFileName": "/OSGI-INF/bulk-config.xml", "xmlPureComponent": true } ], "fileName": "nuxeo-core-bulk-2023.9.10.jar", "groupId": "org.nuxeo.ecm.core", "hierarchyPath": "/grp:org.nuxeo.ecm.core/org.nuxeo.ecm.core.bulk", "id": "org.nuxeo.ecm.core.bulk", "location": "", "manifest": "Manifest-Version: 1.0\r\nArchiver-Version: Plexus Archiver\r\nCreated-By: Apache Maven 3.6.1\r\nBuilt-By: root\r\nBuild-Jdk: 17.0.10\r\nBundle-ManifestVersion: 1\r\nBundle-Version: 1.0.0\r\nBundle-Name: Nuxeo Core Bulk\r\nBundle-SymbolicName: org.nuxeo.ecm.core.bulk;singleton:=true\r\nNuxeo-Component: OSGI-INF/scroll-service.xml,OSGI-INF/scroll-contrib.xml\r\n ,OSGI-INF/bulk-component.xml,OSGI-INF/bulk-config.xml,OSGI-INF/bulk-io-\r\n contrib.xml\r\n\r\n", "maxResolutionOrder": 824, "minResolutionOrder": 128, "packages": [], "readme": { "blobProviderId": "default", "content": "nuxeo-core-bulk\n===============\n\n## About\n\nThis module provides the ability to execute actions asynchronously on a -possibly large- set of documents. This is done by leveraging [Nuxeo Streams](https://github.com/nuxeo/nuxeo/tree/master/modules/runtime/nuxeo-runtime-stream#nuxeo-runtime-stream) that bring scalability and fault tolerance.\n\n## Definitions\n\n\n- __document set__: a list of documents from a repository represented as a list of document identifiers.\n\n- __action__: an operation that can be applied to a document set.\n\n- __command__: a set of parameters building a request to apply an action on a document set.\n\n- __bucket__: a portion of a document set that fits into a stream record.\n\n- __batch__: a smaller (or equals) portion of a bucket where the action is applied within a transaction.\n\n## Requirements\n\nTo work properly the Bulk Service need a true KeyValue storage to store the command its status,\nthere are 2 possibles choices:\n\n- Use `SQLKeyValueStore` this is the case if you are using a SQL database.\n- Use `MongoDBKeyValueStore` this is the case if you are using the `mongodb` template.\n\nYou should not rely on the default `MemKeyValueStore` implementation that flushes the data on restart.\n\n\n## Bulk Command\n\nThe bulk command is the input of the framework.\nIt is composed by the unique name of the action to execute, the NXQL query that materializes the document set, the user submitting the command, the repository and some optional parameters that could be needed by the action:\n\n```java\nBulkCommand command = new BulkCommand.Builder(\"myAction\", \"SELECT * from Document\")\n .repository(\"myRepository\")\n .user(\"myUser\")\n .param(\"param1\", \"myParam1\")\n .param(\"param2\", \"myParam2\")\n .build();\nString commandId = Framework.getService(BulkService.class).submit(command);\n```\n\n\n## Execution flow\n\n![baf](bulk-overview.png)\n\n### The BulkService\nThe entry point is the [`BulkService`](./src/main/java/org/nuxeo/ecm/core/bulk/BulkService.java) that takes a bulk command as an input.\nThe service submits this command, meaning it appends the `BulkCommand` to the `command` stream.\n\nThe BulkService can also returns the status of a command which is internally stored into a KeyValueStore.\n\n### The Scroller computation\n\nThe `command` stream is the input of the `Scroller` computation.\n\nThis computation scrolls the database using a cursor to retrieve the document ids matching the NXQL query.\nThe ids are grouped into a bucket that fit into a record.\n\nThe `BulkBucket` record is appended to the action's stream.\n\nThe scroller send command status update to inform that the scroll is in progress or terminated and to set the total number of document in the materialized document set.\n\n### Actions processors\n\nEach action runs its own stream processor (a topology of computations).\n\nThe action processor must respect the following rules:\n\n- action must send a status update containing the number of processed documents since the last update.\n\n- action must handle possible error, for instance the user that send the command might not have write permission on all documents\n\n- the total number of processed documents reported must match at some point the number of documents in the document set.\n\n- action that aggregates bucket records per command must handle interleaved commands.\n This can be done by maintaining a local state for each command.\n\n- action that aggregates bucket records per command should checkpoint only when there no other interleaved command in progress.\n This is to prevent checkpoint while some records are not yet processed resulting in possible loss of record.\n\nAn [`AbstractBulkComputation`](./src/main/java/org/nuxeo/ecm/core/bulk/action/computation/AbstractBulkComputation.java) is provided so an action can be implemented easily with a single computation\nSee [`SetPropertiesAction`](./src/main/java/org/nuxeo/ecm/core/bulk/action/SetPropertiesAction.java) for a simple example.\n\nSee The [`CSVExportAction`](./src/main/java/org/nuxeo/ecm/core/bulk/action/CSVExportAction.java) and particularly the [`MakeBlob`](./src/main/java/org/nuxeo/ecm/core/bulk/action/computation/MakeBlob.java) computation for an advanced example.\n\n\n### The Status computation\n\nThis computation reads from the `status` stream and aggregate status update to build the current status of command.\nThe status is saved into a KeyValueStore.\nWhen the number of processed document is equals to the number of document in the set, the state is changed to completed.\nAnd the computation appends the final status to the `done` stream.\n\nThis `done` stream can be used as an input by custom computation to execute other actions once a command is completed.\n\n\n## How to contribute an action\n\nYou need to register a couple action / stream processor :\n\n```xml\n\n \n\n```\n\n```xml\n\n \n\n```\n\nSee [`SetPropertiesAction`](./src/main/java/org/nuxeo/ecm/core/bulk/action/SetPropertiesAction.java) for a very basic action implementation.\n\nYou can find more info on how to configure a stream processor in the following link:\nhttps://github.com/nuxeo/nuxeo/tree/master/modules/runtime/nuxeo-runtime-stream#stream-processing\n\n\n## Testing a bulk action with REST API\n\nHere is an example on how to launch a bulk command and get status:\n\n```bash\n## Run a bulk action\ncurl -s -X POST 'http://localhost:8080/nuxeo/site/automation/Bulk.RunAction' -u Administrator:Administrator -H 'content-type: application/json+nxrequest' -d '{\n \"context\": {},\n \"params\": {\n \"action\": \"csvExport\",\n \"query\": \"SELECT * FROM File WHERE ecm:isVersion = 0 AND ecm:isTrashed = 0\",\n \"parameters\": {}\n }\n}' | tee /tmp/bulk-command.txt\n# {\"commandId\":\"e8cc059d-6b9d-480b-a6e1-b0edace6d982\"}\n\n## Extract the command id from the output\ncommandId=$(cat /tmp/bulk-command.txt | jq .[] | tr -d '\"')\n\n## Ask for the command status\ncurl -s -X GET \"http://localhost:8080/nuxeo/api/v1/bulk/$commandId\" -u Administrator:Administrator -H 'content-type: application/json' | jq .\n# {\n# \"entity-type\": \"bulkStatus\",\n# \"commandId\": \"e8cc059d-6b9d-480b-a6e1-b0edace6d982\",\n# \"state\": \"RUNNING\",\n# \"processed\": 0,\n# \"total\": 1844,\n# \"submitted\": \"2018-10-11T13:10:26.825Z\",\n# \"scrollStart\": \"2018-10-11T13:10:26.827Z\",\n# \"scrollEnd\": \"2018-10-11T13:10:26.846Z\",\n# \"completed\": null\n#}\n\n## Wait for the completion of the command, this is only for testing purpose\n## a normal client should poll the status regularly instead of using this call:\ncurl -X POST 'http://localhost:8080/nuxeo/site/automation/Bulk.WaitForAction' -u Administrator:Administrator -H 'content-type: application/json+nxrequest' -d $'{\n \"context\": {},\n \"params\": {\n \"commandId\": \"'\"$commandId\"'\",\n \"timeoutSecond\": \"3600\"\n }\n}'\n# {\"entity-type\":\"boolean\",\"value\":true}\n\n## Get the status again:\ncurl -s -X GET \"http://localhost:8080/nuxeo/api/v1/bulk/$commandId\" -u Administrator:Administrator -H 'content-type: application/json' | jq .\n#{\n# \"entity-type\": \"bulkStatus\",\n# \"commandId\": \"e8cc059d-6b9d-480b-a6e1-b0edace6d982\",\n# \"state\": \"COMPLETED\",\n# \"processed\": 1844,\n# \"total\": 1844,\n# \"submitted\": \"2018-10-11T13:10:26.825Z\",\n# \"scrollStart\": \"2018-10-11T13:10:26.827Z\",\n# \"scrollEnd\": \"2018-10-11T13:10:26.846Z\",\n# \"completed\": \"2018-10-11T13:10:28.243Z\"\n#}\n```\n\nAlso a command can be aborted, this is useful for long running command launched by error,\nor to by pass a command that fails systematically which blocks the entire action processor:\n\n```\n## Abort a command\ncurl -s -X PUT \"http://localhost:8080/nuxeo/api/v1/bulk/$commandId/abort\" -u Administrator:Administrator -H 'content-type: application/json' | jq .\n\n```\n\n\n\n## Debugging\n\nAll streams used by the bulk service and action can be introspected using\nthe Nuxeo `bin/stream.sh` script.\n\nFor instance to see the latest commands submitted:\n```bash\n## When using Kafka\n./bin/stream.sh tail -k -l bulk-command --codec avro\n```\n| offset | watermark | flag | key | length | data |\n| --- | --- | --- | --- | ---: | --- |\n|bulk-command-01:+2|2018-10-11 11:18:34.955:0|[DEFAULT]|setProperties|164|{\"id\": \"b667b677-d40e-471a-8377-eb16dd301b42\", \"action\": \"setProperties\", \"query\": \"Select * from Document\", \"username\": \"Administrator\", \"repository\": \"default\", \"bucketSize\": 100, \"batchSize\": 25, \"params\": \"{\\\"dc:description\\\":\\\"a new new testbulk description\\\"}\"}|\n|bulk-command-00:+2|2018-10-11 15:10:26.826:0|[DEFAULT]|csvExport|151|{\"id\": \"e8cc059d-6b9d-480b-a6e1-b0edace6d982\", \"action\": \"csvExport\", \"query\": \"SELECT * FROM File WHERE ecm:isVersion = 0 AND ecm:isTrashed = 0\", \"username\": \"Administrator\", \"repository\": \"default\", \"bucketSize\": 100, \"batchSize\": 50, \"params\": null}|\n\n\nTo get the latest commands completed:\n```bash\n./bin/stream.sh tail -k -l bulk-done --codec avro\n```\n| offset | watermark | flag | key | length | data |\n| --- | --- | --- | --- | ---: | --- |\n|bulk-done-00:+4|2018-10-11 14:23:29.219:0|[DEFAULT]|580df47d-dd90-4d16-b23c-0e39ae363e06|96|{\"commandId\": \"580df47d-dd90-4d16-b23c-0e39ae363e06\", \"action\": \"csvExport\", \"delta\": false, \"processed\": 3873, \"state\": \"COMPLETED\", \"submitTime\": 1539260607207, \"scrollStartTime\": 1539260607275, \"scrollEndTime\": 1539260607326, \"completedTime\": 1539260609218, \"total\": 3873, \"result\": null}|\n|bulk-done-00:+5|2018-10-11 15:10:28.244:0|[DEFAULT]|e8cc059d-6b9d-480b-a6e1-b0edace6d982|96|{\"commandId\": \"e8cc059d-6b9d-480b-a6e1-b0edace6d982\", \"action\": \"csvExport\", \"delta\": false, \"processed\": 1844, \"state\": \"COMPLETED\", \"submitTime\": 1539263426825, \"scrollStartTime\": 1539263426827, \"scrollEndTime\": 1539263426846, \"completedTime\": 1539263428243, \"total\": 1844, \"result\": null}|\n\n\nOf course you can view the BulkBucket message\n```bash\n./bin/stream.sh tail -k -l bulk-csvExport --codec avro\n```\n| offset | watermark | flag | key | length | data |\n| --- | --- | --- | --- | ---: | --- |\n|bulk-csvExport-01:+48|2018-10-11 15:10:26.842:0|[DEFAULT]|e8cc059d-6b9d-480b-a6e1-b0edace6d982:18|3750|{\"commandId\": \"e8cc059d-6b9d-480b-a6e1-b0edace6d982\", \"ids\": [\"763135b8-ca49-4eea-9a52-1ceaa227e60a\", ...]}|\n\nAnd check for any lag on any computation, for more information on `stream.sh`:\n```bash\n./bin/stream.sh help\n```\n\n## About Nuxeo\nNuxeo dramatically improves how content-based applications are built, managed and deployed, making customers more agile, innovative and successful. Nuxeo provides a next generation, enterprise ready platform for building traditional and cutting-edge content oriented applications. Combining a powerful application development environment with SaaS-based tools and a modular architecture, the Nuxeo Platform and Products provide clear business value to some of the most recognizable brands including Verizon, Electronic Arts, Sharp, FICO, the U.S. Navy, and Boeing. Nuxeo is headquartered in New York and Paris. More information is available at www.nuxeo.com.\n", "digest": "6bc43d75fa011b853f5e4b627aebf5b2", "encoding": "UTF-8", "length": 11329, "mimeType": "text/plain", "name": "README.md" }, "requirements": [], "version": "2023.9.10" } ], "creationDate": 1710762362683, "key": "Nuxeo Platform-2023.9", "name": "Nuxeo Platform", "operations": [], "packages": [], "pluginSnapshots": {}, "releaseDate": 1710762362683, "version": "2023.9" }