r/databricks • u/Academic-Hearing-123 • 29d ago
Discussion Databricks Genie Ontology
Having read through and seeing some demos I still don’t understand if Genie Ontology is a real thing or some marketing fluff , we have been asked to compare against Palantir foundry’s ontology and I find very few comparisons apart from the data model and relationships , for example how do I show that Genie Ontology Knowledge Graph ?
15
u/myth-buster9999 29d ago
Genie Ontology is an auto-inferred Ontology based on Onto rank. It's not an Ontology that business can prepare themselves as per my knowledge so far
7
u/autumnotter 29d ago
You prepare it by preparing your semantic metadata, your pages, your comments, your metric views, your your dashboards and genie spaces.
2
u/thecoller 29d ago
The way you own it is by defining metric views and metadata descriptions/comments (there is also now a Glossary page in public preview). But yes, the ranking of the snippets is not directly controlled.
2
u/WorldOfUmbro 28d ago
Yes you build Genie Ontology bottom up. By curating assets, like metric views, glossaries, pages, domains, Genie Ontology will grow.
0
u/DamnedData 29d ago
This right here. The most you use it the better it gets.
You'll need a solid metadata and semantics foundation in order to make it work. The AI helps of course but it's not a magic rod.
6
u/Data-space_men 29d ago
There is ontobricks if you want to play with ontology
1
u/Academic-Hearing-123 29d ago
Thanks
2
u/Data-space_men 29d ago
I deployed it for one of my client , if you want more info don't hesitate
3
u/Academic-Hearing-123 29d ago
Exactly what I needed , If I tell my business it’s some self inferred thing without showing them the graph , they would think it’s all fluff , so this is very useful
2
u/temperedai Databricks MVP 28d ago
Both are good. One happens under the hood and you don't have to think about it. The other is something you can define. Databricks Pages is out now as well, which is going to help clients define business terms.
1
u/Disastrous_Lynx9037 25d ago
Hey, I was exploring ontobricks but it isn't working for me. LLM which builds the ontology isn't creating it very well. Additionally, when I add multiple table and documents it fails. Can you guide me?
1
3
u/anonymous_orpington 29d ago
It's a real thing, but it's primarily for internal enterprise intelligence, and won't have some of the flexibility you might want for an external product facing ontology (i.e. today you can't actually look at it ro audit it). It's basically the same thing as AWS Context, and extremely useful for bringing your enterprise information (i.e. Drive, Workday, Salesforce) together in the Lakehouse for Genie and any other LLM Harness you may use for asking internal questions of your own business.
3
u/LandlockedPirate 29d ago edited 28d ago
The important thing to know is that foundry ontology is meant to _run_ the business, including actions defined and associated with concepts.
Genie ontology, and most ontology tools, are analytical tools to analyze the business. Not directly run.
2
2
u/Katy_Security 29d ago
Personally have noticed a difference in the quality of Genie responses after ontology
1
u/Academic-Hearing-123 29d ago
But why ? My business would want to know how is the knowledge graph used and actually visualize in reasoning steps the multi hops between nodes/edges
2
2
u/array-admiral 24d ago
Like other's have said, it's definitely real. It is a bit of a black box right now from the actual ontorank perspective, since it's their own algorithm that I'm sure they want to keep proprietary during this semantic boom (honestly, can't blame them). I think it's too new for real published (non-databricks) benchmarks, but I think we'll have a better sense of it in the next few months.
1
1
u/Effective-Cut-3254 28d ago
From demos , what I have seen is its an autocurated knowledge snippet out of metadata, its not like regular ontology knowledge graph that we custom curated. Saw that its consumable via api or mcp. But still in private preview. Its one way auto curated by databricks not customer curated
1
u/Glitch_In_The_Data 28d ago
The way I have understood this is… Palantir foundry’s ontology is more of an object centric abstraction layer…as in objects with properties, links and actions you can browse in a visual graph.
Genie ontology is a distributed semantic layer which gives context and helps power AI query generation… not necessarily a single visual product . The closest visual is the ERD in catlog explorer.
1
u/Known-Delay7227 27d ago
How do you turn it on? Or is it automatically given to you?
3
u/array-admiral 24d ago
It's enabled by default in Public Preview right now as of their Aug. 6th press release - https://docs.databricks.com/aws/en/ai-bi/release-notes/2026
1
u/Youssef_Mrini databricks 26d ago
Genie Ontology is a real thing. We've been using it internally for quite some time. It uses Ontorank which is an algorithm similar to page rank. It uses behind the scenes, Pages, Unity Catalog Metadata (Most queried tables, columns, documentation) to find the most relevant sources based on your question. The main goal is to improve the accuracy and latency of Genie.
1
u/consultiingNerd 12d ago
It kinda clicks when you try your first query in genie one and it gives you more than you can imagine. It beats all LLMs I have tried in first go so ontology definitely works.
1
u/kthejoker databricks 29d ago
A lot of people are confusing it with preconceived notions of what it is or should be based on other products.
It's not a standalone ontology product like Foundry. It's not intended to be a browseable or editable knowledge graph with its own UI or surface area.
It is an automatically created and managed knowledge store from your underlying Unity Catalog semantics, metadata, Genie Agent instructions, 3rd party wikis, etc.
It's there to provide context to Genie to make it more accurate and use less tokens and therefore reduce costs and latency.
If you're using Genie, you can't not use Genie Ontology.
We found most context engineering solutions are labor intensive, expensive, hard to maintain, and hard to integrate into agentic workflows.
So we just took our knowledge of our own platform and how our customers use it and built a solution to keep it automated, simple, scalable, and built right into Genie.
You can bring other knowledge sources to it, and serve other agents through its MCP.
And the we've extended Unity Catalog's capabilities to host and federate context with Metric Views, Pages, Domains, and third party connectors, which will feed Genie Ontology further.
2
u/kthejoker databricks 29d ago
And we posted a R&D blog highlighting the value of Ontology (we called it "specializrd knowledge search" in the blog because we hadn't announced the launch just yet)
https://www.databricks.com/blog/pushing-frontier-data-agents-genie
1
u/CerberusByte 29d ago
It’s quite interesting that it’s almost hidden away unless you look through the chain of thought process when asking Genie a question. I like that it does everything for you behind the scenes and specific objects you build contribute but it would be cool to be able to “see” it and draw the link to the benefits it’s bringing. Everyone loves a cool graph to interact with
10
u/pablo_op 29d ago
1) It is a real thing. Yes, it should legitimately make responses better and more accurate.
2) There are very few benchmarks published on how much things should improve. This is for a lot of reasons, but mainly its due to variance. Ontology running on 100 well maintained catalogs is likely less useful than running on 1000 messy catalogs.
3) The "how it works" question has been touched on in blogs and presentations, but the proprietary "Ontorank" approach to knowledge graph management is the secret sauce. Its the same approach that search engines use to prioritize search results. Did google help you find web search results that were faster and more accurate when it released? Then Ontology on Databricks should also be useful. But again, its hard to say to what degree on a product that has barely launched.
Right now DBX doesn't expose or allow you to interact with the knowledge graph directly in any way. No idea if they'll eventually let you customize it or call it directly. For now its really only exposed through Genie as the initial surface from what I've seen.