We utilize Elastic Search to bring a bunch of data sources together into a large search corpus, which is used to power our core research platform.

We don't generally do a lot of full-text search with Elastic Search. We do a lot of keyword-based searching and a lot of faceted search, and it works really well. We've also had to build custom relevance algorithms based on data that's being stored in the search index. This is more about the algorithm being less about text matching and more about feature matching and relevance on a number of different scales. It's generally worked out really well.

The best feature of Elastic Search is it does exactly what it says. It's really easy to get set up and running and have search running very quickly with basic, out-of-the-box features. It scales very well, and we can do a whole lot with the core feature set before having to move to more advanced concepts. Even then, it performs very well, whether we need to expand into vector databases or decide that the Elastic Search Query DSL doesn't solve our needs anymore and have to go with ESQL or something. It expands and scales really well.

The hosted solution means Elastic Search takes care of the maintenance, which is one of the reasons we chose it. There's been very little maintenance from a data perspective on our side. As we make changes to our database structure, we've had to mirror them into Elastic Search.

We haven't had the opportunity to use the hybrid search with Elastic Search yet. I think there's a place for it in our long-term solution, but we're not quite there yet.

We haven't yet used any AI features built into Elastic Search.

To do what we want to do with Elastic Search, the queries can get complex and require a fuller understanding of the DSL. Once we start to build that understanding, it's another muscle we have, so it's not a bad thing, but it just takes a while to get up and running with expertise for our engineers.

It's not hard to learn how to use more complex things in Elastic Search; it's just a challenge we're going to face.

In my career, I've been using Elastic Search for three or four companies, probably on and off for 10 years.

We've had various very small blips with Elastic Search, but it's never been an issue that was concerning. We have limited infrastructure, so we could go further in terms of our hosted deployment to ensure that some of those things didn't happen. We've simply accepted the level of risk we have.

Thus far, everything seems really good in terms of scalability for Elastic Search. We don't have the largest data set in the world; we have millions of records, single-digit millions, so two or three million records. I feel confident knowing that we could times that by 10 or 100, maybe, and it would still work. The cost would obviously scale, the number of nodes would scale, but Elastic Search would be able to handle that level of scale.

Before I was using Elastic Search and actually before Elastic Search even existed, I previously used Apache Solr and Lucene in my career. The release of Elastic Search way back when was a boon because it was out of the box and did what it said. We've also worked with Pinecone, Amazon's OpenSearch, and essentially Postgres trying to do vector search in Postgres. All of those tools have their place, but if we're doing straight search, Elastic Search is just really the right answer.

The initial deployment of Elastic Search was really straightforward because we used the hosted solution.

We had Elastic Search live and our first initial searches running in our staging environment within a week. We moved into production with our full data set within six weeks.

We had one engineer working on this implementation. That's why it took six weeks.

Elastic Search's pricing is affordable when using the hosted solution through Elastic Search. The pay-as-you-go monthly approach has been nice, and if we scale as a company grows, we'll probably switch to a prepaid model, which will be an even bigger benefit. Having the hosted solution and not having to pay for essentially a DevOps person on staff to manage makes it affordable. We haven't really looked into serverless, which has its own benefits. I think serverless still had some challenges early on, and I wanted to go with something I had previously worked with. The hosted solution pricing fits, but the pricing for serverless also looks really interesting. The self-managed solution is nice from a pricing perspective, but we need the right staff to support it, and we don't have that staff.

We don't use Elastic Search for log ingestion, though I think they have a feature for this.

We haven't worked with anything in terms of Elastic Search integration process for third-party models with interference endpoints.

I'm not using the Attack Discovery feature because we're not using Elastic Search for our observability approach.

We have no partnerships or anything with Elastic Search. I would rate this review as a 9.

Elastic Search is normally used for full-text search where users are fully depending on it for searching by name, address, and similar fields, and we need to gather the data with good latency, so we normally prefer to save it into Elastic Search.

Elastic Search helps for full-text search because we normally use it for keywords and other related terms. If there are keywords and searching requires numerical data and other elements, we prefer RDS over Elastic Search. However, if it is regarding complete full-text search in which we cannot do any kind of indexing and it is very difficult, we prefer Elastic Search.

Elastic Search's best feature is that it is very convenient to save, plus it is schema-less, and it has very good latency and also provides us with different kinds of mapping strategies which allow us to optimize many things according to the data structure. It is a kind of non-structured and structured mix.

The benefits of using Elastic Search are mostly for two to three purposes. For logging, it is very easy to insert the logs into Elastic Search and start searching it using Kibana, and it is very easy to make visualizations over there. The second purpose is that we normally use it for views. If we have searches from the front end with a specific structure, it is very difficult to go to a different table and create the query in the database, so what we do is sync our database with Elastic Search and create a view on Elastic Search which will give us the result in milliseconds. This is how we are currently utilizing it.

Elastic Search has an annoying limitation regarding page size. It has a specific limit for queries on Elastic Search, and the default is ten thousand, and we can increase it. However, after increasing, it can slow down. Pagination in Elastic Search is very slow. If I need to parse one million records saved into Elastic Search, it becomes a nightmare because I need to do the pagination, and it is very problematic in that regard. I need to do ten thousand records and then go to the other page, and when going to the other page, it currently takes much more time than RDS. For specific cases, if we need to do full-text search and searching for one specific word returns less than ten thousand records, it works very well. However, if we go for more than ten thousand, then it creates an issue for us.

It has been almost ten years since using Elastic Search.

Elastic Search receives a stability rating of nine point five; we rely on it.

In terms of scalability, for the managed service, it is very easy, but the scalability aspect is a bit tricky. Scaling up Elastic Search cluster requires a bit of time because of sharding and replications. It takes more time since it needs to copy the data. For example, if we are working on three nodes and adding a fourth node, the synchronization process will occur in the middle, and the higher the data volume, the more time it will take. Scalability is rated around five to six.

Elastic Search's technical support receives a rating of eight.

Previously, we were using the AWS managed cluster on the cloud, but now we have created our own. On the same cloud, we have deployed Elastic Search on our EC2 machines, so it is self-managed, not on-premises. On-premises would be if we give the solution to somebody else, then we would deploy Elastic Search on their specific cloud, but we only deployed it in our system.

I did not go into the deployment part of Elastic Search because it is a DevOps matter. I was in a senior role, so I sent the request and we received it. Normally, it does not take a lot of time if the person deploying is capable; it does not take more than two to three days.

We have about twelve specialists.

I cannot say much about the return on investment part because we normally work on a use case basis. If we find some kind of issue in our database which is currently taking time, then we need to shift to Elastic Search, and it will start giving us very good results. On the cost-saving side, rather than increasing our RDS from a less cluster to a big cluster, we can create a specific separate Elastic Search cluster, and it saves our money on our basic structure while giving us much more performance. I cannot tell you the exact part on how much was saved with the calculation, and I cannot provide the numbers, but it saves our time on the debugging side. Using it on the logs and creating visualization is very convenient for us to search the log and identify the issue as soon as possible. This saves our time, saves the customer's time, and decreases the time to respond and resolve.

Elastic Search's pricing totally depends on the server. Managed services from AWS are used, and we have worked on a self-managed Elastic Search cluster. On the AWS side, it is very expensive because they charge based on query basis or how much data is transferred in and out, making it very expensive. That is why we moved to the self-managed option. In self-managed, it is very easy to handle. We do not think any kind of proprietary Elastic Search solution is required.

Elastic Cloud Serverless is not being used. The GenAI experience with features like agentic AI, RAG, or semantic search is not currently being used. Kafka Streams is being used for log instigation.

Elastic Search has many pros, but the cons of it are that it is not structured, and we need to put all the things which are connected into a single index. Therefore, we cannot use it for our base structure database, but we always use it for supporting purposes.

While part of Careem, there were hundreds of thousands of customers using the solution, and now that in a startup, the clients are no more than one hundred.

Elastic Search requires maintenance. We need to keep it updated because Elastic Search normally launches new features and versions on both Kibana and Elastic Search sides. We need to keep updated ourselves, and also, we need to do maintenance on the storage side. Normally, we use Elastic Search for timelines, saving all the data from beginning to end, so normally the storage maintenance is an issue, and we have to increase the storage time to time, but it is not related to Elastic Search; it is actually related to our use case.

There is lots of support for Elastic Search in different tools like Logstash which we normally use for integration, and there are other tools as well, but it is very easy and not a big issue for that.

The Attack Discovery feature is not being used. Big businesses cannot survive without Elastic Search because it gives us very good visibility and handles our use cases very well. If we need something reliable and trustworthy as a solution, then Elastic Search is the way to go, as it is an integral part of big solutions. The overall review rating for Elastic Search is eight point five.