Monte Carlo Data + AI Observability Platform

Monte Carlo is a data observability tool that can help track data volume changes and flag incidents if there is any unusual activity on a table, data models, or any jobs. For instance, if something unusual happens on an ETL job, it will raise an incident and send alerts via integrated platforms like Teams application and emails.

Monte Carlo also helps in monitoring applications like ServiceNow, Jira, and Snowflake by establishing connectivity with them. The solution is beneficial for various scenarios, such as when sudden deviations from normal data patterns occur. For instance, if a cloud data warehouse like Snowflake experiences an unexpected change, Monte Carlo flags incidents immediately. Its AI agent enhances troubleshooting by providing analytical insights. Besides, integrations with other applications, such as ServiceNow and Jira, ensure an end-to-end alerting mechanism.

On an account level, it builds monitoring capabilities across vast data sources. It also supports triggering alerts via ServiceNow by using webhooks, allowing associates to take immediate action. Recently, Monte Carlo introduced an AI agent to aid in troubleshooting, ensuring only main production layers are analyzed. This backend troubleshooting does not grant complete access to all layers but remains highly effective in problem-solving.

The most valuable aspect of Monte Carlo's observability feature is its automation of the monitoring processes, which eliminates the need for an individual to manually monitor numerous models or tables. It flags issues with precision and ensures proactive resolutions only on the affected components, thereby enhancing efficiency vastly.

Monte Carlo's scalable nature further bolsters its value proposition. Once integrations are established, future model updates are automatically captured without additional setup costs or actions. Given that the data platform's needs perpetually grow, Monte Carlo provides seamless adaptability.

The software manages data auditing and monitoring across platforms like Snowflake with its robust algorithms. By analyzing metadata over an extended period, Monte Carlo's flagging system, based on deviations from historical averages, ensures precise incident identification. Its ability to utilize custom monitors further extends its value, as users can implement logic-based rules and receive targeted alerts.

The introduction of a performance tab greatly aids optimization, visually displaying runtime graphs to identify model issues quickly. Monte Carlo's near perfection in accuracy ensures every flag corresponds to a genuine issue, attested by its consistent performance over time.

Monte Carlo's AI troubleshooting agent, which mimics human oversight through tiered analysis, provides ample support in incident resolution. This ensures incidents are well-documented, analyzed, and tackled despite limited access to all data layers.

While Monte Carlo frequently updates its UI platform, the changes might pose adaptation challenges for long-time users, as the continual evolution is not always intuitive. Additionally, occasional latency hampers efficient access during critical incidents, leading to potential misses of high-priority alerts.

In terms of data accessibility, granting read-level access to non-sensitive data layers would enhance insights for users significantly. Users currently experience limitations in data layer access, such as bronze and silver data layers containing essential business logics.

Sharing metadata with clients could bolster Monte Carlo's analytical capacity, allowing clients to draw deeper insights from shared data.

I have been using this solution for four years.

I have never noticed something which Monte Carlo flagged that was not relevant to the issue. The accuracy is 100% from what I have noticed. I have never noticed any issues where something is actually happening on a data warehouse, but Monte Carlo still flags it as an issue. I have not seen these kinds of scenarios with Monte Carlo while working. The accuracy is consistently 100%.

I can say Monte Carlo's scalability is at approximately 90%. Once we establish the integration in the future, we can enable an option to capture the future models. We do not need to work again and again. Once we establish the connectivity, we do not need to work on adding new tables. It should be a one-time effort. This approach is more scalable. Data platforms will not stop growing from their beginning state. They will always expand. Monte Carlo demonstrates scalability in adopting new models automatically, which should serve organizations well.

We have connected to Monte Carlo regarding a few things. When we are establishing any new connection and face any issues, we reach out to them for technical support.

The technical support is fine. They will respond within two to three hours, but the solution may take some time, ranging from 24 to 48 hours. Technical support is satisfactory from them. Even though the product application team is not that much larger, they are still giving better support.

When onboarding Monte Carlo, you can review the documentation for whatever source you want to connect from Monte Carlo. There you can find more use cases.

When I started working with Monte Carlo, I did not see as many features as currently exist. Previously, the product did not have troubleshooting agents. Also, when I started working, it did not have a performance tab. The performance tab shows performance in a graphical way, allowing me to easily review the model and check the average run time. If there is any unexpected spike that happens for a specific day, I can see that.

For technical support, I would give it eight out of ten. Currently, for my account, we are not giving all layers access to Monte Carlo. We are only giving access to the main golden layer. We are not giving access to the bronze layer and silver layer because they contain business logics.

I am not certain about billing information because that is at an account level, and clients would be aware of billing information rather than myself.

My overall rating for this product is eight out of ten.

I work as a business analyst and I usually see data anomalies in our company's data set, and I also work a lot on Power BI reports to see our performance on the supplier side.

When we receive data from our suppliers to view their performance, sometimes the data is not complete or they are doing backfills of previous data, so we have established some rules in Monte Carlo to monitor these anomalies, and whenever we see something passing a preset limit, we receive an alert.

When I open Monte Carlo, I usually look at my dashboard to see how many alerts we have received in the last few days, but I usually check the alerts in the last month, and I see which rule has received the maximum amount of alerts; then I try to solve it first because the pattern is similar, and then I try to solve other alerts based on other rules.

In my team, it's me who handles those alerts, but we have another team who works on these alerts as well, although they are working on another kind of data set, but in the company, it's used by many people.

The best features of Monte Carlo for my work are the ability to see alerts clearly, how many alerts we have received on which rule and for which country, and there is a feature called investigation query inside Monte Carlo which shows a pre-done analysis, so you don't have to run an SQL query by yourself to do manual checks.

It gives a clear analysis in a large data set, which is very time-saving, so I don't have to run manual codes to verify the data, and it has helped me a lot in saving time and improving efficiency while doing data checks or verifying data. It's a really great tool to explain the anomalies when we see one in Monte Carlo, as we have actual proofs to show to people or to the managers that we are having this anomaly or that data is missing.

There's also an AI feature that is inbuilt in Monte Carlo, but you have to pay separately for that feature, and I used it for quite a while in the beginning, but now my company has disabled it.

The biggest pain point with Monte Carlo is that we have created some rules, but those rules cannot judge everything, and I think the platform is a bit complex for someone new, so it can be more intuitive; a display adoption platform could guide the user on how to use this, like a DAP system. It took a lot of time for me to learn it, and without a guide, a new user would be clueless.

If I could change one thing about Monte Carlo, it would be for the platform to suggest some data quality rules by itself or some algorithms based on the anomalies and the patterns of our anomalies, which would be helpful, and also changes in our rules according to past anomaly patterns. I think that would be good, and they should also improve their support system, as I find it a bit weaker at the moment.

It's been six months since I've been working on Monte Carlo, and it's a really great tool for analyzing data quality anomalies.

Previously, the data checks were performed manually; they extracted data on Snowflake and then did manual verifications on Excel using formulas.

It was not me who implemented Monte Carlo; it was another senior data analyst who implemented it a year ago, but I think it took a few months to get everything up and running.

It needed formal training because the tool is not that easy; if someone doesn't have a data analyst or business analyst background, you have to explain every rule which you have set by yourself, because the rules are created by us, not Monte Carlo; Monte Carlo is just a tool. We put our own rules to govern the data sets, and we literally had to make a guide to help users get to know that platform.

It was not me who implemented Monte Carlo; it was another senior data analyst who implemented it a year ago, but I think it took a few months to get everything up and running.

Monte Carlo saves me roughly 30% to 40% of my time in doing verifications or data quality checks, and related to my team, I'm the only one who uses it in my team, but we collaborate with another team that uses it.

I didn't ask that question in my company; it was a good choice, so it's a very popular tool, which is why my company picked Monte Carlo.

I would like to leave a review for the tool called Monte Carlo, which is a data quality analysis tool.

I think they just changed the layout a little bit, and during my work, I didn't see any changes in the platform except for the AI feature, which I don't use anymore.

Our final goal is to automate every data quality manual check into Monte Carlo so we don't have to do a lot of checks by ourselves.

If I find something bizarre, I go to Monte Carlo and see if it's happening there as well, so it gives me confirmation that the issue is occurring and we can contact our supplier or the country to verify what it is.

Monte Carlo is a really good tool; whenever I do data verification checks for the dashboards, and if I find something bizarre, I go to Monte Carlo and see if it's happening there as well, so it gives me confirmation that the issue is occurring and we can contact our supplier or the country to verify what it is. I would rate this product a 7 overall.

My main use case for Monte Carlo starts with working on data quality related problems, but now it is a very necessary element for our whole data pipeline because it lets us know about the freshness and other data quality related metrics as well, covering all the data quality related metrics, including completeness, correctness, accuracy, and majorly, the freshness part of it.

A specific example of how Monte Carlo helped me with a data quality issue is in our architecture, where we are getting all the data, storing all the historical snapshots in a raw layer, deduplicating it in the second layer, and curating it in the third layer, with alerts and monitors on all layers whenever we move from raw to dedupe or dedupe to curated, checking whether the number of IDs or the number of data is as expected or not, leading to alerts being triggered in the past for the orders table that helped us investigate pipeline related failures.

I often rely on Monte Carlo for our tables that get refreshed in three hours, where it sets up alarms or alerts if the tables do not get updated on their usual trend, such as triggering an alert if a table has not been updated for four or five hours.

Monte Carlo has positively impacted my organization by removing the need for manual monitoring of pipelines, automatically triggering alarms whenever something goes wrong, thus allowing the data engineering team to focus on more positive work, and helping us in manual tasks where after every logic change, we can track data and updates, along with proper catalog maintenance to find out specific information in our data warehouse.

Since using Monte Carlo, the freshness of our data has improved a lot from less than eighty percent to above ninety percent and there has been significant time saved, noting that while we do not keep a precise record of this, there is a steep decrease in time consumed on monitoring and related activities.

The best features Monte Carlo offers include an AI related trend analysis tool that checks the number of records of a certain table or the kinds of records affected by delete, insert, or update operations, triggering alerts if those numbers become unusual and providing a triage solution to investigate specific base tables or parent tables behind specific issues.

The AI trend analysis and triage solution have helped my team by alerting us during manual deletions or update activities and if there is a logic change in the main curated layer; if deletion rates deviate from expected numbers, we receive alerts that we may have messed up with the code, allowing us to check the code logics that we have implemented.

Regarding Monte Carlo's AI capabilities, it offers a tool that provides a mechanism to select or exclude specific parts of the data from the training cycle, allowing companies to adjust incorrect or ambiguous trends in the data, thus showing consideration for governance.

There are some improvements needed for Monte Carlo's code used for migration, which has not been set up well; improving documentation and migration features from other services, along with enhancing historical maintenance and version control on Monte Carlo's code, would greatly help.

In some cases, with multiple tables, the UI sometimes crashes, but it is still the best I have seen so far, making it a great tool overall.

I have been using Monte Carlo in my previous organization for about one year, and here in my current organization, I have been using it for one and a half years.

Monte Carlo is stable, with ongoing feature improvements; while there were initial breaking issues, they are fixed quickly when reported.

In terms of scalability, Monte Carlo handles our organization's multitude of tables and connections well, although there could be improvements in its implementation scalability, particularly with monitors as code.

Customer support was great, with dedicated resources from the Monte Carlo team who assisted with issues during our weekly calls, ensuring we understood specific features.

We previously used the Great Expectations library, which did not offer a solution like AI trend analysis and only provided basic data-based monitoring, lacking features that led us to switch to Monte Carlo.

In the beginning, I found that Monte Carlo took time to learn and understand the metrics and trends we have, but after six or seven months, it has shown great and accurate responses.

We have been tracking our return on investment, which has not been long, but we have saved significantly in time utilization of our resources and in capturing criticalities through this solution.

In terms of pricing, setup cost, and licensing, I rate it a bit high on the pricing side; it is pricey, but given the features and flexibility it offers during implementation, it stands out against specific libraries that are less handy to use, requiring extensive documentation.

Before choosing Monte Carlo, we evaluated Evidently AI, and our existing organizational ties to Monte Carlo influenced our decision-making.

My advice for others considering Monte Carlo is to assess whether their data platform is large enough to benefit from AI capabilities, as smaller scale industries with basic rule-based monitoring might find it a bit pricey. I would rate this review a nine out of ten.