Bundle org.nuxeo.elasticsearch.audit
Documentation
-
README.md
nuxeo-elasticsearch-audit
About
This project provides a backend based on Elasticsearch for Nuxeo for Audit Service.
The idea is to use Lucene / Elastocsearch as storage backend for the Audit trail entries.
Usage of Elasticsearch as a replacement for the JPA based default backend allows to easily make the Audit service scale :
- when there are a lot of entries
- lot of access
- custom logging
- when there are a lot of queries
- reporting
- usage of sync systems like Nuxeo Drive
- when custom attributes on Audit entries are used
How it works
An Elasticsearch based
AuditBackend
is contributed at startup to replace the default JPA based one and it use dedicatedaudit
Elasticsearch index to handle storage and queries.The queries and PageProviders are based on Elasticsearch native DSL : no automatic conversion is done between EJBQL and Elasticsearch DSL.
The orginal Audit service uses a JPA sequence to assign each audit entry a unique id.
In the Elasticsearch implementation, an alternate sequence genaration system is used :
nuxeo-elasticsearch-seqgen
.Data Migration
When
nuxeo-elasticsearch-audit
is deployed it will automatically replace the default JPA implementation.However, if you have previous data inside JPA you have to migrate it.
The migration is done in a Worker that will simply go through all existing JPA Audit entries to store them inside the Elasticsearch index.
The migration is done using a configurable batch size (default is 1000 entries).
An Automation Operation is provided to trigger the migration Work.
Sample call using curl :
curl -H 'Content-Type:application/json' -X POST -d '{"params":{"batchSize":5000}}' -u Administrator:Administrator http://127.0.0.1:8080/nuxeo/api/v1/automation/Audit.StartMigration
NB : Migration of 16 Millions entries on a C3.XLarge AWS instance with default PGSQL setup and default embedded Elasticsearch takes about 3h (1500 entries/s).
Building
To build and run the tests, simply start the Maven build:
mvn clean install
- when there are a lot of entries
-
Parent Documentation: README.md
nuxeo-elasticsearch
About
This project aims at providing Nuxeo bundles to integrate Elasticsearch with the Nuxeo Platform.
The idea is to index/query Nuxeo Documents to/from Elasticsearch.
Building
To build and run the tests, simply start the Maven build:
mvn clean install
Running tests
Using the embedded Elasticsearch
With the default
RestClient
:mvn -nsu test # -Dnuxeo.test.elasticsearch.client=RestClient is set by default
Or with the
TransportClient
mvn -nsu test -Dnuxeo.test.elasticsearch.client=TransportClient
Using an external Elasticsearch
- Start an Elasticsearch using docker:
docker run -p 9400:9200 -p 9600:9300 docker.elastic.co/elasticsearch/elasticsearch-oss:6.5.3
2.a Run the test with
RestClient
:mvn -nsu test -Dnuxeo.test.elasticsearch.addressList=http://localhost:9400
2.b Or run the test with
TransportClient
:mvn -nsu test -Dnuxeo.test.elasticsearch.client=TransportClient -Dnuxeo.test.elasticsearch.clusterName=docker-cluster -Dnuxeo.test.elasticsearch.addressList=localhost:9600
Links
- Administration: http://doc.nuxeo.com/x/UBY5AQ
- Configuring mapping: http://doc.nuxeo.com/x/WxI5AQ
- Overview: http://doc.nuxeo.com/x/iYElAQ
- Open jira tikets
Resolution Order
You can influence this order by adding "require" tags in the component declaration, to make sure it is resolved after another component. It will also impact the order in which contributions are registered on their target extension point (see "Registration Order" on contributions).
Components
- org.nuxeo.ecm.platform.audit.search.contrib
- org.nuxeo.ecm.platform.audit.PageProviderservice.es.contrib
- org.nuxeo.elasticsearch.audit.contrib
Packages
Maven Artifact
File | nuxeo-elasticsearch-audit-10.10-HF25.jar |
---|---|
Group Id | org.nuxeo.elasticsearch |
Artifact Id | nuxeo-elasticsearch-audit |
Version | 10.10-HF25 |
Manifest
Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Created-By: Apache Maven
Built-By: jenkins
Build-Jdk: 1.8.0_222
Bundle-ManifestVersion: 1
Bundle-Version: 1.0.0
Bundle-Name: Audit ElasticSearch backend
Bundle-SymbolicName: org.nuxeo.elasticsearch.audit;singleton:=true
Nuxeo-Component: OSGI-INF/elasticsearch-audit-contrib.xml,OSGI-INF/es-
audit-pageprovider-contrib.xml,OSGI-INF/es-audit-search-contrib.xml
Exports
- Json Export Default Json serialization
- Json Graph Json dependency graph
- Json Contribution Stats Json statistics for contributions
- CSV Contribution Stats CSV statistics for contributions
Charts
-
Raw Data: Json Contribution Stats