Name: Governance in the Age of AI Agents
Uploaded: 2025-07-30T16:30:59.142Z
Duration: 1 h 4 s
Description: Governance in the Age of AI Agents

Transcript for "Governance in the Age of AI Agents": Alright. Good morning. Good afternoon, everyone. I see we have a lot of good evenings. We have a lot of good morning a couple good mornings, couple good afternoons. Well, we're we're happy that you're joining us for this Dataiku live session where we'll be diving into, governance in the agentic age. So we are very excited for you all to join us. Go ahead and leave within the comments where you are watching from. I always love to see where people are watching from. Some people are local. I am based out of New York, New Jersey area. So sometimes I see the New York pop up, but sometimes I see, very far places. Right? London and Frankfurt, Germany, North Texas, Toronto, California or Toronto, Canada. I'm sorry. Charleston, South Carolina, Paris, France. So this is awesome. So we're we're literally watching from all around the globe, and we are very excited for you to join us. So I'll go ahead and start sharing my screen, and we can hop into the content today. So my name is Chad, and we will do formalized introductions in just a moment. But I do have one question for you all before we start. AI agents are here, and they are making decisions. They're handling processes. They're running parts of your business now. But is your governance actually ready for this? Because governing one model is one thing, but governing a fleet of AI agents that are learning, adapting, making decisions autonomously, that's a completely different challenge, and that's exactly what we're gonna be tackling today. But before we hop forward at all, I do have a poll question for you guys, and I also do have some quick housekeeping, topics. So on your screen, you can see the chat. Obviously, many of you are already using it. That's great. We also have a q and a section. So all of your questions, you can direct them there. It should be right next to that chat. Should say q and a. You can tap there, ask any questions. We'll ask them or we'll answer them during the session, but we'll also have time at the end to answer, any lingering questions as well. We also do have a poll section, which we will open up right now. You should see that on your screen now. It should say poll. And the first question that we have for you is I'll actually go ahead and share my screen. How many of you have had an AI model or maybe an AI agent behave unexpectedly in production? So we have a couple different options there for you. We would love to hear your answers. So please let us know. We we would love to get an understanding of where you guys are coming from, as we talk about governance moving forward. So I will keep that poll open for a while. I'll stop sharing and I'll go back to, our slides and we'll actually hop into things. So like I said, we'll go ahead and give a formalized introduction. My name is Chad Coven. I'm a senior technical specialist in product marketing here at Dataiku, and I am joined by Hi. I'm Kirsten Hoogenacker. I'm a senior solutions engineer at Dataiku, based out of Minnesota. I saw somebody else local, so always excited to see a fellow Midwesterner on these big global calls. Yeah. And, you know, counting my past life as a solutions engineer here at Dataiku, between the two of us, we've we've helped tens of hundreds of companies navigate this exact challenge of how do you govern AI at enterprise scale without killing the innovation that makes AI valuable in the first place. So as we hop in, before we dive any deeper into governance, let me give you a thirty second version of who Dataiku is and why we're having this conversation. So we are the universal AI platform, and this means that we unite four critical pieces of your AI strategy. Your people, so regardless of their technical background. Your data, whether it live, wherever it lives. Your technology stack, so you're always using what's best for your organization at any given time. And most importantly for today's discussion is your governance. Right? We handle the entire data to AI life cycle with governance built in and not just added on. So unlike point solutions that require you to cobble together governance after the fact, everything is integrated from day one, and that is gonna be key to what we're discussing today. So this unifying approach is what we take and what we've really seen work for many of our customers to scale to that next level. So before we dive into, any more details, let me also share why governance isn't just something that we talk about here at Dataiku. It's something we actually deliver. Gartner named us a leader in the magic quadrant for data science and machine learning platforms this year for the fourth time. Forrester has reported some serious ROI gains from customers, and what's most relevant for today is what IDC said. They specifically called out that we give organizations the tools they need to manage AI models throughout their life cycle, including regulatory compliance, security, and ethical concerns. So when the industry's top analysts consistently recognize our governance capabilities, it reflects something important. We're not just talking about governance theory, we're delivering governance solutions that work at enterprise scale with real customers solving real problems. So now let's hop into some of these customers. Right? Let's hop into what we've learned working with over 700 of our customers. AI scaling comes down to three critical drivers. You need democratization, so getting AI in the hands of more people. You need acceleration, getting value faster. But here's the thing. Right? You need trust because none of that matters without trust. So you can democratize and accelerate all you want, but if leadership doesn't trust what AI is doing, then everything grinds to a full stop. So why is this idea of trust and governance more important today than it was just three or four years ago? And I think that the answer to that is is pretty obvious. Right? We're in a totally new agentic era. We're not just talking about static models anymore. We're talking about systems that can actually act. Right? Agents are retrieving knowledge. They're calling tools. They're routing decisions. And that power introduces an entirely new risk and a completely new set of requirements. So let's think about it. Agents can approve loans and and route insurance claims. They, can, you know, pull from knowledge banks that are constantly updating. So when something goes wrong, we need to be able to explain exactly what the agent knew and why. One agent calls another, which calls a model, which triggers a tool, and you need to be able to trace that entire chain from data to decision. And what's beautiful is that they're gonna get better over time, and they're already getting better over time using continuous learning. But agent improvement can outpace control without the right monitoring in place. And regulators are expecting auditability and explanations with every single change across your AI system. So you need to be able to understand the rationale, and traceability of your outputs. But here's the thing. Right? So good governance actually increases velocity by preventing rework, enabling safe rollouts, and making audits predictable instead of emergencies. And time really matters here. So every quarter that you delay building proper governance is another quarter of potential risk and technical debt that will compound our time, making it harder to trust your AI outputs. So let's talk about the way that we approach trust in building AI. We see it as three interconnected pillars. You have your AI governance at the top. That's orchestrating and enforcing the rules that align AI with your business objectives. You got responsible AI on one side, making sure that your models are reliable, accountable, and fair. And you have ML ops and you have, you know, AI engineering ops on the other side ensuring that there's smooth operationalization. But here's the key, they're not separate systems. They work together to create trusted AI that actually works in the real world. So let's let's talk about the real world. Right? Agentic governance isn't a one size fits all problem. Because of agents, now it's vital that you have different people in your organization with different concerns, with different responsibilities, and they need different tools. So your design builders, like your data scientist and your AI engineers, they need to answer agent structure and get through approval processes quickly. Then you have your design champions and managers. They need project visibility and risk management. Your IT admins need deployment infrastructure and monitoring. And your governance managers need to know what's running across the entire enterprise at any given moment and ensure everything stays compliant. So one tool for four very different jobs to do. So let's make this extremely concrete. Right? Let's make this a real world example. So let's imagine that we're part of an enterprise insurance company and we're doing insurance claims with models and agents. And it's not just claims. Right? We have agents doing multiple different projects and use cases as part of our enterprise. So you can see what the agents are actually doing, like retrieving, version knowledge, like policies and playbooks, calling tools and models to analyze data, producing recommendations. But enterprise scale governance means having authority and risk limits, enforced, having everything versioned and tracked, having approval, having sign offs, and having continuous monitoring. So now before we actually see how this works, let's understand the hierarchy of governance and how they apply to each user type that we talked about earlier. So let's first start with overhead governance. Right? The the highest level of that hierarchy. This is your governance manager. The person who needs to keep the company safe as AI scales. Making sure that no automated decision risk the bottom line, risk customers, or risk the company's reputation. And their job is to set clear limits, require the right test and approvals, maintain the registry of all AI assets, and enable fast rollback when needed. So we're gonna pop over to Kirsten. And, Kirsten, would you mind if you could hop over to Dataiku and show us how it actually looks like for a governance manager to do this work. Yeah. Happy to. Right. So, hopefully, you see this full Dataiku view, and I'm gonna move all of your beautiful faces out of the way real quick. And in this case, we're working with a life insurance company. And so they might have multiple types of workflow. So maybe beyond just life insurance, they're also looking at different retirement plans or they have other types of insurance that that they're selling alongside. In this case, we're looking at this workflow specifically for the policy management process, of life insurance from end to end. So this can start with all of the processes that are associated with that application process, the underwriting process of, you know, making sure you've got the right types of policies. There's multiple types of of underwriting that might happen. Sometimes certain applicants might be back fast tracked through the underwriting process. Approving and issuing those particular, life insurance policies, making sure that folks are getting paid, and then monitoring those and potentially having, like, a next best offer for the customers who are already purchasing these life insurance policies. So we'll go through, the step by step process as the governance risk manager. I wanna make sure that I have a full understanding of what's happening across this entire life cycle because it often requires multiple teams. So the application team that's, you know, driving different, you know, ML classifications using data extraction to better understand who's applying might be a completely different team from those who are running the next best offer. That might be more of like a marketing team. But I really need to understand this process from end to end. So here I can see the different descriptions that are associated with each step of the process. And maybe what's a little more important is I can see each project that's associated with, each of these different steps, what models they're using. I think more and more often, we're gonna see that these kind of multileveled hierarchical projects where you might have multiple agents that are also calling these traditional machine learning models, that you'll see here in green. I might need to see a a different view of this that is more stepwise where I get to see each step of the process, which projects are associated with it, and as you saw all the different models and agents, as well. I also think that we're kind of in this stage where, you know, we don't have our full process. We're just getting started with this particular governance workflow, and I, was notified that there's a new project that I actually need to take a look at and actually apply these governance policies to. So I'm gonna head over to our governable items here. Actually, I'll go to the home page so you can see it. There are model registries in data I could govern. We have a GenAI registry. What we were just looking at was a type of business initiative, and we'll kind of get into this, a little bit later, but it allows you to kind of group your different projects together and really assess priority for each of those different projects. So earlier today, I got a notification that there was a new claims project that I needed to take a look at. In this case, I can go ahead and and click on this. I can get some details so I can see whether or not we're hosting an LLM. I can see what kind of embedding models we have on here, and I think I wanna take a closer look at this particular project. And I I actually wanna assign a governance policy to it. So we'll go ahead and govern this item, and we've actually created a couple of different templates for governing projects that have agented workflows in them. In this case, it's that application workflow. We wanna make sure that we have all of these steps, as a part of that process. And this particular claims project is gonna be a part of our issuance, initiative. So this is when we're actually looking at, you know, assigning and issuing those claims. And then I have some additional templates that I can use. So the bundles are going to apply to the policies that I have when I push something into production. So I'm gonna go ahead and, we're just gonna use our our standard template for that. There are different templates that I can assign to these models and agents. So I'm gonna assign our application, template here and then for the model version. So as Chad mentioned earlier, having these specific version level controls is gonna be really important for me. So we might have, you know, three different agent versions as we've kind of iterated through this process. And we know that traditional machine learning models like class ifications and regressions also go through multiple versionings, as well as we kind of update these different projects and models. So we've got a separate, template for these particular types of projects here. And when I hit govern, I'm basically applying a level of metadata on top of this project, which now allows me to see that, in our workflow. So I wanted to go back real quick to our governable items, and we're actually going to take a look at this this project itself. So as the risk manager, I can see some of the things that are happening alongside this this project itself. I might need to have some visibility into the different datasets that were used. Maybe I wanna take a look at the the schema or the lineage to make sure that, everything is packaged and safe. You'll see I have a nice timeline here. So Dataiku, really does a nice job of trying to automate some of the general governance practices, as a part of just building projects, as a whole. So this is actually where the project is being built. And so there are some other things that I wanna do to make sure that I've actually got the right prioritization of of projects here. So I'm gonna hand it back to Chad real quick here. Yeah. So that's great. Right? We're able to see the overhead governance, sort of level. So now let's move down a level, right, to the program level. So this is where your design manager, who is maybe an IT team leader or a data science director, they might come in. And their challenge is they need to build reliable compliant processes that actually accelerate AI, deployment while reducing any sort of compliance surprises. So they're the ones providing templates, rollout steps, shared tools, and they're tracking all projects and defining what done actually looks like. So, Kirsten, let's go ahead and hop back into Dataiku and show how this works from their perspective. Yeah. So in this case, we've got that same type of flow here. We've got a couple of ways that we can actually view the projects that this manager is responsible for in the govern note itself. You'll see all the different projects, the different business initiatives. And one thing that's kind of cool is we can actually take a look at things like different models that might be, you know, needing to take these Gen AI models to to another level. So we know that there's gonna be some level of originality and autonomy that these individuals might be, looking into in terms of, like, their agents. So is the content that they're creating original that that can cause some risk? How autonomous are these chatbots? So this provides me a way to prioritize the projects that I I I wanna take a look at. And there's another view. So I I can also take a look at this from more of like a timeline. So projects that are kind of in an exploration phase, in a validation of rollout stage, and projects that are are delivered. So I've got a full view of all the projects and teams that I manage, which is nice to have this in one place. And then, as I kind of dig down a little deeper, I might need to get a little more granular in the way that I'm assessing the work that my team is doing. And I'm actually gonna do this in our design environment. So this is our our governance framework. Right? We can see this at the high level, and then I can start kind of digging in, a little deeper here. So on the homepage of Dataiku, you can see all the projects that I have access to. I've got my, you know, different teams who are gonna be working on different projects. I can see my recent or favorited items. It's really easy for me to go ahead and and find the projects that I need to look at. If I wanted to create different workspaces, I could. Workspaces just allow us to kind of rebucket some of these, projects together. I've got a data catalog. We'll kinda get into some of those lineage pieces because I do think that inherently, as we talk about governing agents, we're also talking about governing AI. We're also talking about governing data. It's all kind of wrapped up into the the same same type of, in here. Alright. So let's head into this claims agent project. I'm gonna go back to the home page for this real quick. Again, I did peek let you peek at this earlier. We've got a timeline here. We've got the flow. I have a really nice description. So as I come in, I can understand what, is going on in this project. I can take a look at the Wiki page, which is actually going to, link to different parts of the project. Again, the descriptions help us understand the risk that might be associated. It helps us to make sure that we're applying the right types of regulation to each of these different projects. And then I'm gonna need to, you know, get a little nitty gritty sometimes with what's going on in our workflows. And so I just wanna make sure that I've got, you know, some view into the stepwise process to each of the different datasets that are being used. Here I can see that we've got, an investigation agent that's pulling from a number of different datasets. So I've got that visibility into the lineage of these projects, what they might be doing, and I might have some additional questions for my team who's building out these projects individually. So that kinda gets us into the next, role here. Yeah. Thanks, Kirsten. And and so we've we've seen governance from these two different levels. Right? So going from that overhead governance, right, seeing it from more of an enterprise wide sort of scope and then going down to a bit more of a project level. So now let's actually see how it looks for builders themselves. Right? How it's built into everything that they do within Dataiku. So these builders are gonna be your data scientists, your AI engineers, etcetera, etcetera, who need to really ship improvements and and net new features fast. But with safety built in by default. So not bolted on later. So their workflow is we have to version everything, run the checks, produce traceable evidence, with each action, and request approval for deployment when the AI product is ready. So, Kirsten, let's see what that looks like from a builder's day to day experience. Excellent. Okay. So as a part of being one of these builders, I suppose for for some of you who are new to Dataiku, this is what the workflows look like. We've got different datasets that we can apply recipes to. Today, we're gonna focus mostly on some of these LLM recipes and also creating agents. So I can actually take a look at the building blocks of agents themselves. So every agent is built from a number of tools that get called. In Dataiku, we've got a bunch of different types of tools that we can, get into visually. You can go ahead and custom code your own tools. There's Python tools that you can go ahead and add, as well. So it is kind of a quick way for me to iterate through creating agents. As a part of any of these tools, we'll just take a look at this industry risk. You'll notice that we actually have this, LLM mesh is what we call it as a part of me being able to select different LLMs and or agents that I would wanna call as a part of this particular tool. I can go ahead and describe what this agent is going to be doing. So this is kind of pulling in that, hierarchical view, and I can go ahead and just write out some prompts. What I like about the tool building in Dataiku is that it's really easy for me to run different tests on these tools as I'm troubleshooting and trying to understand how and why agents are making decisions and where I might need to go through and do some additional work. So as a part of the tools, you know, these are kind of the individual building blocks of the agents themselves, which I can actually pull up through this agent here. And if I go ahead and oh, that one was, I can oh, why do we see let's see. I'm gonna go back to the home page real quick. There was this one here. So there's this investigation agent, and it is version controlled. So, again, I've got that available to me as I'm iterating through this agent process, and I can go ahead and select different tools to add to my agent. So in this case, we're gonna get some business rules. We're looking at entity thresholds, taking a look at different business rules that I might wanna test out. And Dataiku provides a way for me to go ahead and, as I mentioned earlier, go ahead and run quick tests on my agent itself. So I do get the full response data. I get the logs, and I can actually look at the traces of these tests. And what that means is I can actually understand why the agent is making certain decisions. So we can get some of that content here, or I can go ahead and look at this in another view. So in our traces explorer, this is actually a pretty neat way for me to, a, troubleshoot and, b, track over time what questions are being asked of the agent. It's really clear from an auditability standpoint. So I can go ahead and send automation scenarios or messages when when things aren't necessarily working the way that I would anticipate. I also get some cost estimations. So understanding how many tokens so from, like, a cost governance standpoint, this is also built in to the the Trace Explorer. So you get the visual look at all of the different nodes that are part of the agent decision making, and then I also get some information about how the agent made these decisions, and why it might have responded from a specific way. We can look at it in a few different views. So, you might wanna take a look at the timeline view. We can see all the different portions that are a part of this, and I can actually also explore these in-depth as well. So being able to kind of expand, can trace, search for certain terms if I'm looking for them as well. So lots of ways for me to both track how that agent is performing, why it's making the decisions that it's making, which I think is, really important from a government standpoint. And then also making sure that I can run some sort of automation on this downstream. So Chad had mentioned that version control is important to me. And so there's many ways that Dataiku automatically creates these versions for us before we, push these into production, which is kind of this a whole another level of of version control. The first is that we have, get, like, version control. So I can see every change that has been made to this particular project, all the different commits that were a part of this in this sense. So those are gonna be really individual changes. And then if I go into our agents here, we've got this reflection agent, and I can see that there are two versions on it. So as I'm iterating on these agents, Dataiku will automatically create these versions when we publish them to our broader flow, which then kind of builds into the versioning of the project itself. So in Dataiku, there's, you know, we applied this governance layer to this particular project, which can actually prevent its deployment into our production environments if I don't have the right sign offs and approvals. So in this case, I might wanna create a bundle. And within this bundle, it's just kind of like a snapshot of the project. I can see all the different datasets, managed folders, etcetera. And I also get this government status. So before I try to publish this for deployment, I can see that it is currently being governed in Dataiku and that the status is not validated. And so there are probably some checks and balances that I haven't completed as a part of this process. So when I try to publish this on the deployer, I'm probably, not going to be able to actually push that into an, a higher environment, which is great. Right? So when we think about the governance, we wanna make sure that we've got the right validation. So as an individual builder, I might need to send this to, you know, an architectural review board before pushing this into production, or I might need to make sure that my project is appropriately qualified. And all this kind of built into this full blown process within Dataiku itself. So, there are just multiple levels of being able to kind of govern this as as a builder, make sure that I've got the right version control. It's nice because I don't necessarily have to track all of that individually by myself. Dataiku really aids me in some of those governance policies, making sure that I'm adhering to the appropriate, you know, compliance regulations that might be a part of building out this particular claims project. Right. So I think the other thing I wanted to mention on this, we've got our version control. We talked about two calls, and then being able to bundle this for for production. As I mentioned, I won't be able to move that in there until we've got the right sign offs and approvals, which takes us back to that governance process and being able to, make sure that we're appropriately moving through these steps, one at a time. Alright, Chad. I think I get to hand it back to you. Alright. I appreciate it, Kirsten. So we, before I hop into the rest of it, we we have a ton of questions, and we really do appreciate all of them. We'll actually save many of them for the q and a section, which will come up in just a moment. I have a couple more slides to get through. Before I hop back into those slides, I would love it if we can go ahead and answer this next poll question. I'll go ahead and throw it up on the screen. How long does it currently take your team to deploy a new AI model? Not just the model. Right? It could be an AI application, any sort of AI system, to production. So go ahead and answer that question for me in the poll section. I will go ahead and stop sharing the poll, start sharing my screen again, and feel free to continue to answer that poll question as I finish up here right before we get to our q and a. So we just saw governance from those three separate angles, enterprise oversight, program management, and daily development. And what makes this work is the integration. It's not just governance that tech that's tacked on, it's woven into how agentic AI gets built and deployed. So policies and predictable compliance that speed up development, not slow it down. You have trust without those bottlenecks, and you have complete visibility across your AI projects and productions. And the result is you scale AI confidently because governance actually accelerates instead of being a constraint. So to give this, a a bit more weight, let me leave you with a real example of what this looks like in practice. So, Ora is a Dutch insurance company, and they use this exact approach to transform their insurance claims process. They went from 40% automated claims processing to 80%. Model development time dropped from one year to two months, and their customer satisfaction scores rose so dramatically that their pet insurance claims became their most highest rated service. And most importantly, for today's discussion, they have bias assessment built into every customer impacting use case in their fully EUAI Act compliant. And this is all due to choosing governance that accelerates rather than constrains with Dataiku. So with that, I also recommend that you go to our website. You check out this customer story. It's a very good read. You can find it. Just type in Ora Dataiku on Google and it should be the first, link there. So with that, we can go ahead and open up the discussion for questions. I will not leave this slide up. You can actually see our faces for this. So there are many, many questions, within the chat, and I want to go ahead and address a couple. I saved some specifically for, this moment. So one question was, how do we define agents? And I think and I can go ahead and take a stab at it. And, Kirsten, you can, give your 2¢ as well. But I think the way that we define agents here at Dataiku, really is about we try to make it as simple as possible. It's about decision and it's about acting. So it has to be, some sort of generative system. Right? Utilizing, most likely, an LLM on the back end. Whatever LLM you would like to choose from the Dataiku LLM mesh, which is that LLM gateway that allows you to use basically anyone that you can think of. So utilizing that to make a decision based on your data, maybe based on an ML model that you're connecting to, maybe based on a knowledge bank, and then being able to actually take action based on that decision. So maybe it's updating, some sort of dataset with a prediction from that model. Maybe it's invoking another agent, and having more of a multi agentic chain. Maybe it is actually going into ServiceNow or going into Jira or going into Snowflake Cortex and taking it further than just kind of giving you some sort of reply in a chatbot. So that's, like, the twenty to thirty second version of it. We could probably talk about it for fifteen, twenty minutes. But, Kirsten, go ahead. Yeah. I think I think you did a there are so many good questions in here. So I'm like, also, I think that's a good summary of of agents, I would say, that they have to be able to make some sort of decision or leverage some kind of action based on, the prompt that's that's that it's running through. I do think that they do require LLMs, right, to be able to process what's coming in. So I'd say that those are the three pillars of of defining an agent in this case. Yeah. So, moving forward to the next question, what are the type of trainings and setup options offered to new clients? So for new customers, we have many different services that come with Dataiku. A few of those are trainings themselves with our expert data scientist. Right? They're they're teaching you the platform, but also walking you through how to make your first high value use case. So those are, a few of the services that we give. We also have some free options like the Dataiku Academy. That's a great way to learn the platform. You can find that online. Just go ahead and type in Dataiku Academy because I I cannot think of the URL off the top of my head. But that's a great platform to start to understand how we see, data science and AI, and the utilization of GenAI. So that's where I would start. And, the link is in the chat. Thank you, Kirsten. I thought this question was very interesting from Carl. In your platform and we can kinda take this question by question, Kirsten. In your platform, where are risk mitigation, recovery actions, or contingency plans to address when risk occurs? I'll just read the rest of them and then we can talk about them. Also, where are alerts for different risk events for AI agents or models in your platform? And with the lineage tracking, does your platform specifically track and manage the access of sensitive or confidential data? So, Kirsten, I'll actually take I'll actually take the last one. We'll we'll go from bottom to top. So in terms of lineage tracking and working with sensitive or confidential data, we have many different integrations within Dataiku. We also have some native ways that you can do it. So for one is data quality checks. So you can utilize data quality checks not just for the health of a dataset, but also to make sure that we're not utilizing sensitive data or confidential data given a specific use case. So that's one way that you can utilize, you know, that you can make sure that sensitive or confidential data isn't going through certain sort of workflows. Another way is we do have GDPR integrations. So you can actually understand what, you know, metadata is being utilized. Is it, sensitive or not? What groups should have access to what datasets? You can put different datasets into different safe data collections within our data catalog. So that's just a way to kind of store and tag and log what, datasets, you actually want to use for what given, maybe, business initiative. So you can put locks on on those. Right? So that's to answer that very last question. That middle element. Oh, go ahead. I wanna add one more thing to that question that I think it's at Wayne's question as well who's who's talking about just, like, data tagging. Can we kind of cover this all at once? Are you okay with doing that, Chad? Yeah. Yeah. Let's do it. Okay. So, there are multiple ways that we can kind of tag this data in Dataiku. You can actually manually tag it. With any of these, we can also take a look at things like like lineage. There are automatic metadata tagging that we can also apply, to the the schemas. So within this, it'll create both column level descriptions and dataset descriptions here that you can apply and and use as a part of the the data lineage. And then I also think it's really important that we take a look at something like PII data. So Mhmm. At the connection level in Dataiku, so when you are setting up these individual connections, you know, we really think about data connections as very similar to an LLM connection. Usually, you've got some level of AD group that's gonna have some access to specific datasets. Similarly, we're assuming there's gonna be that type of control over the LLMs that you are allowing, your teams to use. So if we take a look, for example, at our OpenAI, instance here, I can create specific security requirements. Right? So I can give it to anybody on the instance for Dataiku. I can further lock it down by group. But it's also going to require, see. So I think we've just got our API key, but you can have people SSO in to each of these. You'll see I can select specific models. We actually have PII detection as a part of this, as the guardrails that we, are leveraging. So Dataiku will kind of read what's happening within each of these connections and make sure that, you know, you're as you are working through these projects, the builders in this case aren't having to think about it. It's gonna be automatically tagged to keep that data safe and secure. Yeah. And I think the the last thing to speak about when tagging is you can have, what we call macros, which are just, you know, sort of instance wide automations, that automatically tag certain items. So in our case, we we have an internal instance that automatically tags, I believe it's, like, the author as well as, like, the dates and a couple different things of, whenever you create a new project. Right? So you can set up those automated tags, to work on datasets, really to work on any item within Dataiku. So datasets, recipes, models, agents, whatever you choose. So there's there's many ways to approach data tagging and and and, tracking access of sensitive and and confidential data. To answer a few more of your questions in terms of alerts, you can always utilize, what we call scenarios, which is our automation, sort of approach. So you can set up alerts within scenarios so that when, a model or let's say an agent is performing not how you envisioned, you can set up an automation scenario that tracks that model or tracks that agent, their performance, and then you can set up a reporter, which will then email you or Slack you, x y z, you know, when something is off. Right? For for a human to come back into the loop. So this is really important to us. We we really believe in keeping humans in the loop and Kirsten showing, what that scenario suite looks like. So that that's a way that you can continue to have, risk kind of rein in reined in by that human in the loop. And then also with scenarios, you can create some sort of risk mitigation or recovery actions, based on the that bad performance from an agent or a model. Yep. Alright. Let's go ahead and scroll up. We do have a couple more questions. So one of which is, if I were to build this, which team would I have to assemble or require to build, to help build AI, AI engineers, etcetera, please advise. Kirsten, I don't know if you have, any insight on that. So I think it really depends on the project. Right? So to to me, it's kind of qualifying, you know, oftentimes when we see these full workflows, you want somebody from the business who understands the actual use case. What's nice about Dataiku is that it is a platform that the business can get in and and have the understanding of what's going on stepwise throughout the process. You would likely need somebody who understands agents from, like, a a risk perspective. I think it's really important that we kind of build in some of the qualification. I I think you can do it with really lean teams. You can also do it with really broad teams. And I I I don't know if there's a clear advisory on this. I do think that it's really use case specific. Agreed. Agreed. So so yeah. It it definitely is use case specific. Right? Like, the it's the answer that everybody that everybody says. It depends. But be because of that, you know, you can really approach it several different ways, especially in Dataiku. Because like we said, we unify people and it doesn't really matter about your technical experience to to create the things that you want. Alright. So there is a a question here. Is there an option or workflow in Dataiku similar to Mistral library or CHAT JPT library to extract the information from documents with citations and traces, I think the library option is different than a RAG solution. So I think the way that I would see it within Dataiku and, Kirsten, yeah, you can go ahead and oh, actually, I'm sorry. Let me share my screen. Is what we're looking at right now is sort of a rag solution. And from my understanding, what you're saying here is it's less about kind of uploading the documents to then get some sort of retrieval knowledge from the agent or from the LLM. But instead, this is more so about, being able to upload those documents in more of a quick seamless way to not just necessarily get retrieval information, but to actually utilize in in your different agent outputs. That's what I think you're getting at. And if you were to use Dataiku, yes, you could use the knowledge bank, right, for sort of that approach. But then, also, if we hop over to, Dataiku agent connect, which we'll just go ahead and kind of showcase this screen. If you wanted to upload a document here and have an LLM, utilize it or have an agent utilize it, you totally could. Right? So, I think in that way, you could have both that library experience but also have that knowledge bank vector store experience as well, just to make sure I understood your question. Alright. The next question that we have is, is it also possible to check why an agent is taking the decisions it does? So, Kirsten, I'll go ahead and and hand that one over to you. Yeah. So to me, that's that's under that that trace explorer. Go ahead and pull that up real quick. And it would be in the notes, and so you'd be able to helps if you share your screen. So within this, we got all the questions that have been asked of this particular agent. And then what you're seeing is the the decision tree of of how the agent arrived at the specific answer that it did. And there could be multiple things that happen as a part of this pathway. Right? So it could be making a decision that it needs to pull a specific tool. It might need to make a decision to respond in a specific way. And each of these different nodes that you're seeing on the screen is gonna have some level of description of what's happening at each of these decision points. And so you can search these if you'd like, but oops. Sorry. So, basically, we're we're running through different tool calls here. There are different sources that are here. We can see what it's actually, looking at, the items that are being returned here. In this in this case, it's it's pulling out records. And then you've got different structured tools that might be leveraged. And so you can, note by note, kind of run through and see what certain tool calls are are coming up. This is just a dataset lookup. So kind of, it does have that explainability of understanding what decisions were made and why certain tools were called. And so all of that would be here within that Trace Explorer. Yeah. And, even if we want to take that a little further and, Kirsten, I can show my screen real fast. Yep. This is just something that I personally do sometimes to understand my agent traces and and their behavior. Mhmm. So for example, right, I I have an LLM that is just used to understand traces in general. And what's great about Dataiku is you have every single element. Right? We try to make sure it's not a black box. This is a white box approach. Everything that we utilize within platform, you have the ability to take further or customize in any sort of way. So for this instance, I have my trace over here on the left hand side of my LLM contacts. In this case, I gave it the question of, can you give me customer information on a specific customer? And I'm giving a prompt to an LLM saying your task is to make in, the log in plain English so I can understand what the agent is doing. So this is kind of a nice way that you can, this is a nice way that you can really look into your traces in more of a plain English sort of way, to be a companion to that Trace Explorer. So in this case, it's telling me what exactly happened. Right? What was the question that was being asked? How it decided what it did? Right? So it decided that it needed to look up this specific dataset and then why this answer. Right? So there's many different approaches that you can take to understand why your agents do things. Trace Explorer is the first place that that, you know, we would probably point you to. But you can go as deep or, as high level as you want. Okay. So next question. Like I said, we have a ton of questions which we really love. So thank you guys so much. Very quickly, I saw that in the governance section, there is the possibility of creating projects that take into account the EUAI Act. Can the platform be used to create compliant applications in health or automotive, for example? Thank you. So I will just pop into the governor real quick. And oh, wait a second. Here we go. And so there are different templates that you probably saw in here. One of them is the UAI Act. And so this is just looking this is built by we have a government solutions team that has a number of different templates. So I think we've got different templates on GXP. I know that we have some other standards that are kind of built into, this this government node. And it's this one in particular is looking at the EU AI Act. So it's basically as we kind of, go through each of the steps, It walks through each step of the EU AI Act to look at things like design choices, you know, referencing those articles and basically allowing us to go through and edit, answer the summary, upload any supporting documents for each of these different, parts of the EUAI Act. Similarly, you'd be able to build out whatever custom workflows you might need as a part of your governance framework. So that life insurance workflow that we looked at today was something that we built out custom, for this particular industry, particular industry just kind of as a way to to showcase the work that we needed to. So it is completely customizable to meet those needs that you might have. And in addition, Dataiku has a solutions team that has existing templates for, different industries, but then is also there to help you build out whatever custom workflows your governance, compliance, risk teams might require as a part of moving as a part of project qualification, getting things to production, making sure that you have, the right types of review documents available as a part of the process. Awesome. So the next question that we have is do you support MCP? I will say this now. Not yet. It is on the road map, so it should hopefully, be coming, you know, sometime soon. We will keep you guys updated through our through our product updates and our web pages and whatnot, but, that is something that we we are definitely looking forward to. In terms of utilizing MCP type of things, you can create your own custom agent tools, within Dataiku, which can basically serve as, your own internal sort of MCP connectors. So, for example, we have a Jira tool, we have a ServiceNow tool, we have, a Google Calendar tool, many tools that are connecting out to external platforms. Next question that we have, is Dataiku Academy free to access? Yes. It is free to access. We also do have a free trial of Dataiku. I believe it's a fourteen day free trial. So you can go ahead and head to our website, start up that free trial, and then you can go ahead and do Dataiku Academy using that free trial. And Dataiku Academy itself is free forever. So, we don't plan on changing that. Is auto tagging via external ontology, is that possible too? Yes. It is. If you have some sort of external way or rules based system that you would like to tag things, you can implement that system into Dataiku, and, you know, it'll exist in Dataiku, and tag those different items that you're looking to, tag. Alright. Can you pull in data security and classification information from Purview? So I don't believe you can yet, but that's something that we will definitely keep a lookout for. We still have wow. We still have a ton of questions. These are probably the most questions I've ever seen on a webinar. Before we hop into the rest of them, just to make sure that we don't, go over, I do want to share with you all real quick. If you want to get in contact with us, with Dataiku after this webinar, should see that on your screen now, Feel free to contact us. You can head to our web page, dataiku.com. And from there, there is a contact us button. It'll have you fill out a short form, and somebody from the Dataiku team will get in contact with you. You can also go straight to [email protected]. Go ahead and send us an email. We'll receive it, and we'll be in contact with you that way as well. So just in case we're not able to answer all of your questions, we would love it if you can, reach out to us. Okay. So we have about two minutes here. We're gonna answer, just a couple questions. I'm gonna go ahead and pick, some good ones. This one's a pretty quick one. This is a basic question. Can you use the Dataiku framework to run real time systems or near real time? And the answer is yes. You can, do real time API deployments from Dataiku. You can deploy them using Dataiku. You could deploy them using, you know, AWS, using Snowflake, using Google, GCP. So we have a ton of ways that you can make sure that you can do real time deployment. Another question is, can you handle running together concurrently two levels of multi agent systems? One is high performance and the other one is high confidence, trustworthiness, and handle, an overall real time mechanism to switch from one to the other depending on some control or monitoring status. So the answer to that is yes. I think of I have a kind of a decent example. We are running low on time, but the answer to that is yes. Yes. You can run concurrent agents. One can be high level, one can be high performing, one can be more of a validation agent. You can make as many agents as you want. You are not limited to the number of agents. And then each agent can also utilize different, LLMs if you'd like. So with that being said, we are coming up on time. We're wrapping up in just a little bit. So we wanna thank you so much for joining us for this Dataiku live session on agentic governance. Like I said, if you have any questions, please contact us. Go to our website, and we will see you guys at the next Dataiku live. Thank you so much. Thanks so much.