Upgrade Kibana from 5.6 to 6.x

I have been running ELK stack 5.6.4 version on Kubernetes hosted on AWS. When my team asked for an upgrade on Kibana for new visualizations and dashboard. Its not a Kibana upgrade alone. Because Kibana 6.x expects X-Pack to be installed on elasticsearch and its a best practice to keep Elasticsearch and Kibana on same version. So i have to upgrade my elasticsearch cluster to 6.x before upgrading kibana.

Okay !! But will the existing indexes on elasticsearch 5.x will work on 6.x.
Yes..
But the index created for kibana 5.x will be “.kibana” and for kibana 6.x version the index is “.kibana-6”.

So how do we migrate ?

Lets begin with upgrading elasticsearch.
Before starting, make sure you have the backup of your elasticsearch indexes, mappings, templates, etc..

https://www.elastic.co/guide/en/elasticsearch/reference/5.5/modules-snapshots.html
As my data nodes are pointing EBS volume, I have taken the EBS volume snapshot.

Elasticsearch Upgrade
I have used the elastic search images from quay.io, where i have found more stable Elasticsearch docker images.
https://quay.io/repository/pires/docker-elasticsearch?tab=tags
So i have upgraded my elasticsearch to 6.4.2 where X-Pack plugins are preinstalled and disabled.

You can create your own image by downloading the elasticsearch and plugins manually with a trial X-Pack license or paid one.

Once your elasticsearch cluster is up and running, make sure all your indexes are green. If not, make sure the nodes are running properly.

curl -X GET “localhost:9200/_cat/indices”

Now you can upgrade your kibana to the same version of your elasticsearch.
Once your kibana is up and running, you should possibly get the kibana index out of date error.

Now its time to fix your kibana index. Replace localhost with your elasticsearch hostname on the below curl commands.

  1. Set .kibana index to read-only:

curl -X PUT “localhost:9200/.kibana/_settings” -H ‘Content-Type: application/json’ -d’
{
“index.blocks.write”: true
}‘

2. Create .kibana-6 index

curl -X PUT “localhost:9200/.kibana-6” -H ‘Content-Type: application/json’ -d’
{
“settings” : {
“number_of_shards” : 1,
“index.mapper.dynamic”: false
},
“mappings” : {
“doc”: {
“properties”: {
“type”: {
“type”: “keyword”
},
“updated_at”: {
“type”: “date”
},
“config”: {
“properties”: {
“buildNum”: {
“type”: “keyword”
}
}
},
“index-pattern”: {
“properties”: {
“fieldFormatMap”: {
“type”: “text”
},
“fields”: {
“type”: “text”
},
“intervalName”: {
“type”: “keyword”
},
“notExpandable”: {
“type”: “boolean”
},
“sourceFilters”: {
“type”: “text”
},
“timeFieldName”: {
“type”: “keyword”
},
“title”: {
“type”: “text”
}
}
},
“visualization”: {
“properties”: {
“description”: {
“type”: “text”
},
“kibanaSavedObjectMeta”: {
“properties”: {
“searchSourceJSON”: {
“type”: “text”
}
}
},
“savedSearchId”: {
“type”: “keyword”
},
“title”: {
“type”: “text”
},
“uiStateJSON”: {
“type”: “text”
},
“version”: {
“type”: “integer”
},
“visState”: {
“type”: “text”
}
}
},
“search”: {
“properties”: {
“columns”: {
“type”: “keyword”
},
“description”: {
“type”: “text”
},
“hits”: {
“type”: “integer”
},
“kibanaSavedObjectMeta”: {
“properties”: {
“searchSourceJSON”: {
“type”: “text”
}
}
},
“sort”: {
“type”: “keyword”
},
“title”: {
“type”: “text”
},
“version”: {
“type”: “integer”
}
}
},
“dashboard”: {
“properties”: {
“description”: {
“type”: “text”
},
“hits”: {
“type”: “integer”
},
“kibanaSavedObjectMeta”: {
“properties”: {
“searchSourceJSON”: {
“type”: “text”
}
}
},
“optionsJSON”: {
“type”: “text”
},
“panelsJSON”: {
“type”: “text”
},
“refreshInterval”: {
“properties”: {
“display”: {
“type”: “keyword”
},
“pause”: {
“type”: “boolean”
},
“section”: {
“type”: “integer”
},
“value”: {
“type”: “integer”
}
}
},
“timeFrom”: {
“type”: “keyword”
},
“timeRestore”: {
“type”: “boolean”
},
“timeTo”: {
“type”: “keyword”
},
“title”: {
“type”: “text”
},
“uiStateJSON”: {
“type”: “text”
},
“version”: {
“type”: “integer”
}
}
},
“url”: {
“properties”: {
“accessCount”: {
“type”: “long”
},
“accessDate”: {
“type”: “date”
},
“createDate”: {
“type”: “date”
},
“url”: {
“type”: “text”,
“fields”: {
“keyword”: {
“type”: “keyword”,
“ignore_above”: 2048
}
}
}
}
},
“server”: {
“properties”: {
“uuid”: {
“type”: “keyword”
}
}
},
“timelion-sheet”: {
“properties”: {
“description”: {
“type”: “text”
},
“hits”: {
“type”: “integer”
},
“kibanaSavedObjectMeta”: {
“properties”: {
“searchSourceJSON”: {
“type”: “text”
}
}
},
“timelion_chart_height”: {
“type”: “integer”
},
“timelion_columns”: {
“type”: “integer”
},
“timelion_interval”: {
“type”: “keyword”
},
“timelion_other_interval”: {
“type”: “keyword”
},
“timelion_rows”: {
“type”: “integer”
},
“timelion_sheet”: {
“type”: “text”
},
“title”: {
“type”: “text”
},
“version”: {
“type”: “integer”
}
}
},
“graph-workspace”: {
“properties”: {
“description”: {
“type”: “text”
},
“kibanaSavedObjectMeta”: {
“properties”: {
“searchSourceJSON”: {
“type”: “text”
}
}
},
“numLinks”: {
“type”: “integer”
},
“numVertices”: {
“type”: “integer”
},
“title”: {
“type”: “text”
},
“version”: {
“type”: “integer”
},
“wsState”: {
“type”: “text”
}
}
}
}
}
}
}‘

3. Reindex .kibana into .kibana-6:

curl -X POST “localhost:9200/_reindex” -H ‘Content-Type: application/json’ -d’
{
“source”: {
“index”: “.kibana”
},
“dest”: {
“index”: “.kibana-6”
},
“script”: {
“inline”: “ctx._source = [ ctx._type : ctx._source ]; ctx._source.type = ctx._type; ctx._id = ctx._type + \”:\” + ctx._id; ctx._type = \”doc\”; “,
“lang”: “painless”
}
}‘

4. Alias .kibana-6 to .kibana and remove legacy .kibana index:

curl -X POST “localhost:9200/_aliases” -H ‘Content-Type: application/json’ -d’
{
“actions” : [
{ “add”: { “index”: “.kibana-6”, “alias”: “.kibana” } },
{ “remove_index”: { “index”: “.kibana” } }
]
}‘

Reference: https://www.elastic.co/guide/en/kibana/current/migrating-6.0-index.html

Give a few mins for kibana to read the index. Go back and check your visualizations and dashboard.

Thank You !

An IT Infra Architect - Cloud Architect -- DevOps Architect. Works mostly on Opensource, a solution provider.