Ddmit
1.环境
- Elasticsearch 7.1.1；
2.安装
./bin/elasticsearch-plugin install ingest-attachment
然后重启各节点。
3.单个附件处理
3.1 创建索引
为单个附件处理创建个索引：
curl -XPUT 'http://10.47.0.96:9200/data_archives_attachment'
3.2 创建管道流
curl -H 'Content-Type: application/json' -XPUT 'http://10.47.0.96:9200/_ingest/pipeline/single_attachment' -d'
{
"description" : "Extract attachment information",
"processors" : [
{
"attachment" : {
"field": "data",
"indexed_chars" : -1,
"ignore_missing" : true
}
}
]
}'
3.3 创建索引文档
curl -H 'Content-Type: application/json' -XPUT 'http://10.47.0.96:9200/data_archives_attachment/_doc/1?pipeline=single_attachment' -d'
{
"filename" : "ipsum.txt",
"data" : "dGhpcyBpcwpqdXN0IHNvbWUgdGV4dAo="
}'
3.4 索引数据
curl -H 'Content-Type: application/json' -XGET 'http://10.47.0.96:9200/data_archives_attachment/_doc/1?pretty'
返回结果如下：
[inspur@localhost conf.d]$ curl -H 'Content-Type: application/json' -XGET 'http://10.47.0.96:9200/data_archives_attachment/_doc/1?pretty'
{
"_index" : "data_archives_attachment",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"_seq_no" : 0,
"_primary_term" : 1,
"found" : true,
"_source" : {
"filename" : "ipsum.txt",
"data" : "dGhpcyBpcwpqdXN0IHNvbWUgdGV4dAo=",
"attachment" : {
"content_type" : "text/plain; charset=ISO-8859-1",
"language" : "en",
"content" : "this is\njust some text",
"content_length" : 24
}
}
}
4.多个附件处理
4.1 创建索引
为单个附件处理创建个索引：
curl -XPUT 'http://10.47.0.96:9200/data_archives_multi_attachment'
4.2 创建管道流
curl -H 'Content-Type: application/json' -XPUT 'http://10.47.0.96:9200/_ingest/pipeline/multi_attachment' -d'
{
"description" : "Extract attachment information from arrays",
"processors" : [
{
"foreach": {
"field": "attachments",
"processor": {
"attachment": {
"target_field": "_ingest._value.attachment",
"field": "_ingest._value.data",
"indexed_chars" : -1,
"ignore_missing" : true
}
}
}
}
]
}'
4.3 创建索引文档
curl -H 'Content-Type: application/json' -XPUT 'http://10.47.0.96:9200/data_archives_multi_attachment/_doc/1?pipeline=multi_attachment' -d'
{
"attachments" : [
{
"filename" : "ipsum.txt",
"data" : "dGhpcyBpcwpqdXN0IHNvbWUgdGV4dAo="
},
{
"filename" : "test.txt",
"data" : "VGhpcyBpcyBhIHRlc3QK"
}
]
}'
4.4 索引数据
curl -H 'Content-Type: application/json' -XGET 'http://10.47.0.96:9200/data_archives_multi_attachment/_doc/1?pretty'
返回结果如下：
[inspur@localhost conf.d]$ curl -H 'Content-Type: application/json' -XGET 'http://10.47.0.96:9200/data_archives_multi_attachment/_doc/1?pretty'
{
"_index" : "data_archives_multi_attachment",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"_seq_no" : 0,
"_primary_term" : 1,
"found" : true,
"_source" : {
"attachments" : [
{
"filename" : "ipsum.txt",
"data" : "dGhpcyBpcwpqdXN0IHNvbWUgdGV4dAo=",
"attachment" : {
"content_type" : "text/plain; charset=ISO-8859-1",
"language" : "en",
"content" : "this is\njust some text",
"content_length" : 24
}
},
{
"filename" : "test.txt",
"data" : "VGhpcyBpcyBhIHRlc3QK",
"attachment" : {
"content_type" : "text/plain; charset=ISO-8859-1",
"language" : "en",
"content" : "This is a test",
"content_length" : 16
}
}
]
}
}
参考：
