2.5.1 Update Existing Index
Impulse does not support row level updates. You can overwrite partitions within a specific data range and add new data to existing index. Here is an example of how this works:
Assume you have an existing index in Impulse with the following rows:
If we have the following dataset that we want to append to existing index and update the existing partitions within the period 2019-01-01 and 2019-01-03:
The resultant data after the combine and overwrite of partitions within the period 2019-01-01 and 2019-01-03 will be:
Notice that the partitions within the date range 2019-01-01 and 2019-01-03 are replaced by the the new dataset and outside this date range the data are appended to existing index.
Here are the steps to combine and overwrite the existing partitions.
- Follow the steps of ingesting files to a datasource in a warehouse
- In the step 2, select the Upload Mode as “Combine & Overwrite: Combine old data with new data and overwrite” option from the dropdown list (see Figure 2.5.1a below)
- In the partition overwrite period from and to fields, fill out the data range that you want your new data to update to.
- Follow the remaining steps of ingesting data into Impulse.