In the What happens when you index a document section, we discussed that when you index a document, the refresh process creates a new segment. Since writing the segment to the disk on every refresh is very expensive, the segment is only written to in-memory file system cache. When certain conditions are met, a process known as Lucene commit writes all the files (segments) in the memory to a physical disk.
If a node crashes before the files in memory are persisted to the physical disk, the data in the file system cache is lost, and any uncommitted changes are also lost. But Lucene commit is very expensive and cannot be done after every operation. To solve this problem, Elasticsearch introduced transaction log, which is a write-ahead log:
During an index or delete operation, the document is written to both the memory buffer and the transaction log. The index/delete operation...