Video: Modernize Your Governance for Agentic AI at Scale | Duration: 3584s | Summary: Modernize Your Governance for Agentic AI at Scale | Chapters: Introducing AI Governance (2.96s), AI Agent Trust (202.215s), Defining AgenTek AI (343.735s), AI Governance Risks (618.565s), Governance Policy Framework (966.93s), Model Governance Examples (1399.635s), Agent Governance Policies (1567.9349s), Dataiku Platform Overview (2062.21s), Operationalized AI Governance (2138.425s), Q&A Implementation Strategies (2223.4148s), Agent Development Practices (2365.895s), Governance Pillars Expanded (2577.26s), Agent Supervision Standards (2688.52s), Oversight and Risks (2843.8599s), Governance for Agents (2940.265s), External Compliance Challenges (3075.605s), Governance Responsibility Considerations (3234.9502s), Conclusion and Farewell (3550.55s)
Transcript for "Modernize Your Governance for Agentic AI at Scale":
Hey, everybody. Thanks for joining us today. We're gonna talk about modernizing governance for Genetec AI at scale. Pretty excited about the session today. I wanted to give everybody a minute to kind of pile in here before we kick off, so feel free to say hi in the in the chat. We'll be monitoring as we go through here. So today, we will be, discussing what it takes to govern the full AI life cycle. So it's gonna include models and data and AI agents, the governance for all those items. And we'll talk about how organizations are designing scalable governance programs to align, you know, the governance with business strategy, ultimately, so you can keep innovation on track without losing control. That's the goal. I think we've got several folks here in the chat, so we can go ahead and kick off. Before we get started, some housekeeping items. Number one, this webinar will be recorded. So, it'll be available on demand for viewing afterwards. And two, don't forget to ask questions using the q and a tab on the right of your screen. So we'll try to answer throughout the webinar, and we'll also have some q and a time at the end to dive a little bit deeper. So with that said, let me introduce myself and our speaker today. So I am, Christian Casiville here at, Dataiku. I lead content marketing and product marketing. Been an open source data science space for several years. It's really kind of my my passion. I love to do it. And with me today is Kevin Petrie, the VP of research at Bark, and he leads the data management practice and writes about topics such as AI and data integration and data governance. For thirty years, Kevin has, decipher what technology means to practitioners as an industry analyst, as an instructor, a marketer, and a services leader, and as a tech journalist. So Kevin built the data analytics services team for EMC Pivotal in The Americas and EMEA and ran field training at the data integration software provider, Attunity, which is now part of Qlik. So he's a frequent public speaker and coauthor of two books about data management, and Kevin most loves teaching data and AI leaders about evolving strategies, tools, and techniques to capitalize on the value of data. And I am super excited to have Kevin here to share his expertise with us. Thank you, Kevin, for joining us. Thank you, Christian. Great opportunity. So before we kick off, I just wanted to, make sure, you know, for those not familiar, Dataiku is the universal AI platform that brings together your people, data, technology, and governance. So organizations ultimately can create and and control analytics models and AI agents. So this, of course, is a topic that's near and dear to us when it comes to governance of all the things that are in that tech stack or and all of your data and all of your models and all of your AI agents. We wanna make sure that, you know, you have kind of a centralized control and visibility that you're gonna need to operationalize these things and ultimately get value. So, hence, this is why this is a good topic for us to, to chat about today. Now on that front, before we get started, I thought it would be great to ask a quick question, which we will touch on in a little bit in the presentation too. So I'd love to compare the results here. I'd love to gauge where this group is at with regards to AI and trust. And so before I'll stop sharing so I can share the poll here, I'm gonna open it here. And so the question is pretty simple. Would you personally trust an AI agent to send an email on your behalf? And so maybe if I'm gonna qualify that further, I'll say, like, it's not gonna give you a a human in the loop step. You're gonna tell it to go do something, and it's just gonna, like, send the email. We'll give folks a a couple of, a couple of seconds to answer. You know, obviously, lots of attention and hard work are going today to make agents, like, technically feasible. Can they send the email? Yeah. Absolutely. But it'll be interesting to see how much these groups trust them to do something basic like this, today. Very interesting. Okay. Cool. Thank you, thank you for all the responses. I will, I'll close the poll for a second. So we've got, a third yes and two thirds no. That's actually, that is pretty interesting to me. And we'll we'll talk about how that compares to some of the other surveys, that we've done on this in just a minute. So I appreciate you all sharing. I'll bring back up our slides here. So, at this point, I'll go ahead and hand off, to Kevin to kind of dive in a little bit to our topic today. Kevin, it is all yours. Fantastic. Thank you, Christian. Hopefully, folks can hear me and see me. There we go. Really pleased to have the opportunity to talk to you about a very important topic. I've always been, sort of a fundamentalist when it comes to data governance and and risk mitigation, I suppose, as part of my DNA. But we're in an at a stage in the innovation cycle for AI where it matters more than ever. There was the big gong, the big bang moment in November 2022 when Chatt GPT debuted on the world stage and really dazzled everyone in terms of what's possible with these probabilistic models. That sparked, as we know, an incredible investment boom that continues an incredible adoption wave. But now we're at a point where the disillusionment of some of the early failed projects, which always happen with any technology wave, and a realization that you have to have these clean inputs, and you have to have governed guardrails and safeguards, especially with probabilistic models and agents that put them to work. So the notion of governance of AgenTic AI is more important than ever, and it's rightfully getting c level and board level attention, which I think explains why we have so many folks on the call today. Let's start by, defining AgenTek AI. We find at Bark, we do a, a fair amount of survey work, so we can get pretty granular in terms of measuring adoption trends and best practices. We find that about a third of companies say they're in some level of production with agentic AI. I suspect a good chunk of that is agentic features within existing commercial tools that they already have. Still, that shows astonishing rates of adoption, given that the notion of an agent and agentic AI really was not mainstream until early twenty twenty four. So we're moving ahead quickly. If we define what agentic AI means, we have these agents, which are autonomous or semi autonomous applications that are going to respond to human prompts or commands often through a chatbot interface. They will, based on those commands, plan and execute sequences of tasks. They might delegate actions to tools and models, oftentimes multiple tools and multiple models, which can lead to complex workflows. They'll collaborate with one another. If we get this right, you could have marketplaces of agents that transact with one another and converse with one another. They consume data, from platforms and from SaaS applications. Model context protocol is a great, lubricant to help in those interactions with, source data sources. Ideally, these agents are going to start to reflect on the suitability of their decisions, their actions, and responses, and learn and iterate over time so they become self teaching. Now that's a very futuristic view, but it's a view of what good might look like in 20 to 30, if we can get this right. We're not there yet. A lot of these agents are very rudimentary, and they should be because of the risks. So Christian started with an excellent poll question. And you'll be pleased to know that you are echoing concerns, with executives that responded to this, this survey from Capgemini that came out recently. You can see that over the last year, enthusiasm for AI and the potential of AI agents in particular has moderated and there's now rising concern about the risk associated with it to the degree that most executives don't believe the risks outweigh the or believe the risks outweigh the rewards, the benefits right now. And in fact, a majority would not trust an AI agent to send a professional email on their behalf, similar to the majority of folks that dialed in here. So that's natural. I view that as healthy maturation and, a real awareness of risk. We will get there. We've had prior innovation waves, the the steam engine, the PC boom, the Internet build out. In each case, you had this powerful new technology, and it took companies time to figure out how to use it responsibly and build it into their processes in order to accelerate productivity gains. So we'll get there, but we need to have a real awareness of the risk. I thought is an interesting data point. The Global Risk Advisory Council, ranks on an annual basis the top 10 risks to company reputations. The number one risk this year is misuse of AI. The number two risk on it as a side note is association with Elon Musk because he's a perhaps a bit of a wild card. But AI misuse is very much getting a focus as both a potential reward, but also a risk. So we're here to talk about mitigating risks in order to realize the rewards. Most organizations are familiar with data governance, and most of them have some semblance of a data governance program in place that probably includes a lot of the folks on this call. Data governance has been a challenge for thirty or more years. And, unfortunately, it's not a point of strength for most organizations. If we look at where we are with data governance, there's a a significant, level of challenge with data quality, with lineage, regulatory compliance, govern access to data. And so I think those are some of the things that have been plaguing traditional analytics for some time, and it certainly applies to AIML. If we apply that to the notion of AI projects, you can see that a third of companies don't trust the inputs they're feeding to AI models, and 41% don't trust the outputs. So that's a problem. We need to we need to fix that in order to make Agenik AI safe. It's not easy to do because traditional governance controls are no longer sufficient. Traditional governance controls apply to tabular datasets. They apply to SQL, based databases, and they're based on rules and deterministic rules, deterministic inputs and outputs. And now when you get into AI governance, you get into unstructured inputs or semi structured inputs, and you use of words and so forth so that evaluations become much more subjective. And that subjective element can lead to risk and complexity. So what could go wrong? Let's let's drill in a little bit here. A lot of things could go wrong. This is not an exhaustive list, but I think these are the primary primary risks that organizations face. In the data domain, we've seen risks, five specific risks for some time. Factual inaccuracy related, of course, to data quality, issues, inconsistencies, gaps in data, null values, and so forth. Privacy breaches are, an ongoing concern as we see from headlines and, and as we've seen from a regulatory focus with GDPR in The EU, with CCPA in California, and by extension, the rest of the country. There's, of course, the concern about bias due to skewed data samples and, inadvertently exposing intellectual property. There's a need to protect that. And of course, noncompliance is a significant risk. Organizations need to be pretty diligent in order to demonstrate compliance with, as I mentioned, GDPR, CCPA, and then a whole range of vertical specific regulations as well. There's HIPAA, of course, which applies to the privacy of health patient data, which can be relevant to a variety of verticals. Models compound those risks because they might exacerbate what's happening at the data level realm. They also introduce new risks such as toxic output, inappropriate, inflammatory, derogatory content. You can have, a need for explainability. So black box logic is a challenge, especially with Gen AI models, because even the folks that build this stuff cannot quite articulate exactly how it arrives at a certain piece of content. And then semantic inaccuracy. Are you striking the right tone? Are you understanding if you have a chatbot that's interacting with a customer or a partner in a supply chain? Are they understanding the appropriate, tone to strike, the appropriate words to use with those individuals? Now agents are taking actions. They're making decisions, and they're making basing taking actions based on the outputs of these models, both traditional machine learning models and Gen AI models, they could make misguided decisions based on faulty outputs. They could take actions that actually damage the business. So if you have a a model that starts to misinterpret why a customer is angry and automatically takes action that makes the customer more angry, you can see how that could spiral pretty fast. Another big risk here, of course, is subversive intent intentions. And this is something that, I don't think has gotten enough attention. OpenAI and Anthropic, are are both pretty diligent about, articulating publicly the risks of the models that they're building. I personally think Anthropic's a little farther along in terms of mitigating that and really aiming for safe AI. But the specific paper that I think is very telling is from OpenAI, which is about models, reasoning models, having a tendency if they perceive that they're being monitored very closely by humans, they have a tendency to deceive humans, to try to hide their actions and actually subvert human intentions so that they might go rogue and, and perform actions that they were not part of what the humans intended. So those those are some interesting things. And what OpenAI said was was kind of startling in my view. They said, we want you to be aware of this risk and good luck, because extensive monitoring is gonna cause the problem. It's it's unusual that you have governance actually causing the risk as opposed to mitigating it. So all this adds up to increasing levels of uncertainty and complexity. And, there's a a a very thoughtful piece in Harvard Business Review, that came out in June. I'd recommend folks reading it. The the title is organizations are not ready for the risks of a JEDTIC AI. And this individual has, done a lot of consulting with a lot of GenAI adopters and says, if you just look at the the points of failure, among other things, you've got one, model that could be deriving, insights from three source datasets, and then they start to interact with other models. And then that fits into agents as part of workflows, and you have different agents working with one another. That's a lot of point of failures points of failure, and you can have this cascading effect of errors or issues that come up. So it's important to learn to walk or crawl before we walk and walk before we run. So a monitored governance program can really start to mitigate a lot of these risks, so so that you can get things down to an acceptable level, and you can start to realize some of the positive outcomes of initial incremental projects. The critical policies, from my view, fall into the same three domains, data, models, and ages. So governance policies is where now we're gonna define the objectives and the principles, of this of this governance element. The rules are going to define how to enforce them. Use this data quality observability tool to do this. And then the standards are gonna provide the technical structure formats and things like that. So if we look at data, I think three critical policies, not necessarily all of them, need to be data stewardship, authorization of access and usage, and then auditing and documentation, again, to demonstrate compliance with, external regulations, but also internal policies. Models, have critical policies need to have critical policies related to transparency and explainability, responsible output, aligning with, safety and responsible AI, objectives, and then continuous oversight. You need to be very proactive and vigilant in monitoring what these models are doing, and in particular, the outputs that they're handing to agents. In the agent domain, we have some critical policies that are required as well. High level policy for safe decisions. Making decisions that are within established guardrails, and, and don't alienate customers or or otherwise, hurt human relationships. Accountable actions, so that you have, individuals, not just agents, but individuals that are held accountable for, issues that arise. And then, of course, overall permission, permissible behavior, and some of that gets to the subversive intentions that I talked about. Okay. So now I can unpack this a little bit more in each of the three domains. If we if we look at what good looks like with data governance, let's let's unpack the policies, the rules, and the standards in these three areas, stewardship, authorization, auditing, and documentation. From a stewardship perspective, you wanna ensure data stewardship by assigning specific stakeholders to oversee the data life cycle. Could be data engineers, could be data stewards, application developers to the degree they're consuming data as well. And you want to assign those stakeholders and and from a rule perspective, start to define exactly how they're supposed to inspect and curate data. And I think a lot of these supporting controls will be to use, for example, data observability, data quality observability tools to spot issues, inconsistencies, null values, and so forth. And, and and follow established measures for remediating that. And then the standards can be to define the format of the data to define the remediation measures. So that it's like a fire drill. When something happens, you know exactly what to do. And, and, establishing the technical standards as well. Now from an authorization perspective, it's critical to have these policies that authorize your data preparation, your usage, and your consumption procedures. You wanna specify the permissible users, who they are, what team they're part of, the applications they can use, the datasets they can use, and also the time periods that are allowed. So this gets down to role based access controls. And so the standards can define the actual APIs and the actual role based access controls that can be used in order to help enforce the policy of authorization. That's critical. That's been in place for some time and it needs to apply to not just the data elements, but also the act the agents and the models. Auditing and documentation. Nothing new here, but, it's critical to ensure compliance with regulations and laws. And you want to, from a rule perspective, codify your processes for audits and reporting. And make sure that there's an agreed frequency, scope, and format of these reports and these, these audit, exercises. So now, if we go to, there we go, models. Model governance similarly has three critical policies, transparency and explainability, responsible outputs, and continuous oversight. So with transparency and and explainability, we're talking about inspecting and validating the model inputs and the model outputs. So the model inputs might be features that go into training, and you being used in inference for traditional machine learning. They also could be, source data for a RAG workflow, retrieval augmented generation, that, where you retrieve data from an established, agreed proprietary dataset and inject that into the prompt in order to increase the likelihood that a Geni a model gets things right. So you want to make sure that you have clean data. You wanna make sure that you have clean outputs as well. And so from a role perspective, you're gonna wanna document and measure how inputs influence outputs, for example. So there's SHAP and LIME are two different techniques that are used to quantify just how data samples or features impact model outputs for traditional machine learning. There are also ways in which you're going to want to, make sure the the outputs go travel lineage and trace back to the inputs. Now standardized, the standards, of course, would be to specify In this case, you wanted to use SHAP, you wanna use LIME, you wanna have that lineage view, and you want to register all your models alongside the participating inputs within a catalog, within a model registry. So that the right stakeholders, data scientists, machine learning engineers, and so on, have the right visibility into all these elements. Responsible outputs depend on preventing biased, toxic, or harmful model outputs. And here, evaluator models, become critical. And so evaluator models might include natural language processing or traditional machine learning. Different things that can look for telltale signals, of issues coming from Gen AI models. So a Gen AI model might produce output that strikes the wrong tone or misreads the sentiment of a customer email, as I mentioned before. Evaluator models can help, gauge whether the sentiment is appropriate to the conversation and to the customer. You're gonna want to apply those evaluator models, establish filters to remove risky toxic content, and then kill switches to have a human and shut something down and intervene where it makes sense. And the standards would be to define exactly how you're scoring, the suitability of GenAI content and, what are the thresholds and what are the corrective actions. Continuous oversight simply means you're being very vigilant. So you wanna maintain vigilance with vigilance with production monitoring because there can be drifting inputs, there could be change in business circumstances. There could simply be x factors that you didn't see in, in models because you don't have the right visibility into their development. You wanna periodically audit flagged inputs and outputs, and specify the frequency and the format that you're doing that. Again, this isn't an exhaustive list, but I hope it's a sufficiently granular set of examples to give you a sense of how to address risks in these three domains. So now let's look at what it looks like with agents themselves. Yes? I I did wanna I did wanna jump in on the model governance piece real quick if I could. Yeah. Please. Because this is an area that we've worked extensively, with lot of our customers and, you know, we have several, what I would consider, good examples of our model governance amongst our customers at Dataiku. And so maybe I thought it'd be helpful for the audience just to hear what we're seeing with that with that. So particularly with our large finance customers, you know, they have both, like, sensitive data and increased regulatory scrutiny, which makes it, of course, like, really critical for them to have these things, transparency controls, traceability, and auditability. And so maybe to give a specific example, we recently worked with a data team that was attached to the commodities arm of a global investment bank, and they were having problems or, I would say, maybe some difficulties with manual monitoring and reporting on just, like, a few dozen models that they had in production. And, really, they wanted to find a way to, like, be that easier, but also to scale to more use cases, which they were finding they weren't really able to do. And so with some of the things that we have in Dataiku, they were able to basically get those models into a single registry, which helps with the transparency piece. And then they could apply custom workflows that match their specific regulatory requirements. So it helped with the continuous oversight for, their unique situations, so just generally. And one one cool thing that they did also was within Dataiku, we're able to build a visualization, kind of looking at the model monitoring outputs, which they were able to use to simplify basically the audit process. They were kind of always audit ready because of those visualizations and dashboards that they built. So, ultimately, you know, of course, that makes it easier for them to monitor their existing models, but it also enables them to scale, like, new use cases while still maintaining these three pillars, which I think is pretty important especially if you are in, something like health care, if you're in finance, those types of industries where it's this is even more important to you. Having those kind of systems in place, are super super useful. And so I just wanted to echo that, yeah, these pillars are what we're seeing our customers apply as well. That's great. That's a great example. And, I agree completely. What what we find is that, within those regulated sectors, that oftentimes are making use of structured data, which is a little easier to govern in a lot of cases. Finance and health care are two great examples. They, they have a a saw a more solid foundation to start because they're more accustomed to answering audits. They're more accustomed to ensuring the accuracy of their data and then using it as as kind of lifeblood in the way they already interact with customers and so forth. So similarly, I'll talk in a few slides about what AI leaders are, how we define those. They're folks that have these cross functional AI programs. And, FinServ and health care companies are often within that mix, especially FinServ. So that's a great example. So agent governance makes things interesting because now we're getting into making the decisions and taking the actions based on model outputs, outputs from Gen AI models, from machine learning models, and so on. The three big policies are safe decisions, cannibal actions, and permissible behavior. The safe decisions, require restricting your decisions to legal and ethical, what I'll call green zones. Models can go in a lot of different directions. They're proactive. They're like eager 16 year olds who get 1,600 on their SATs. Not a lot of common sense, but eager to please and and be proactive. And you wanna make sure that you have established guardrails that they're going to act within. The rules would be to enable alerts and specific overrides for safety. So identifying model outputs and then actions or decisions that are crossing a line and defining exactly what what the intervention procedure is. And then the standards, of course, could identify who the authorized parties are, humans and agents, what at what authorized actions are, and what the limits are for those. It might be a certain transaction level if it's a shopping agent. It might be a certain time period if it's a different kind of agent. Accountable actions require ensuring the traceability and control of actions. You wanna have human approval of high impact tasks. There's an interesting survey that came out from Iconic, an investment firm. They surveyed 300 executives with, software companies whose livelihood is based on automation and AI. And their number one favorite governance control was human oversight. That was that governance control, beat beat all the other go governance controls. And so I think if if companies that are basing their livelihoods on automation still view humans as the safest way to ensure governance, you wanna make sure you have a human in the loop for the important stuff. From a standard perspective, you wanna map those actions back to models, to logic, and inputs. And then for permissible behavior, you want to anticipate and identify subversive behavior. There are NLP evaluator models that could kick in. There are some different ways to gauge sentiment so that you could start to understand if a model starts to go rogue and behave in a subversive fashion the way I described earlier. You wanna apply adversarial tests to agent outputs and standardize the metadata, the measurement procedures, the measurement, methods for all this, and, of course, the intervention procedures as well. I see a lot of questions coming in, which is fantastic. And I think we can we can answer those at the end if that if that works for folks. So I'll step back here and just say that we've talked about three domains, data, models, and agents. Each of them has their own life cycle associated with them. Data engineers manage a data ops life cycle. Data scientists, machine learning engineers, manage a model ops life cycle. Broadly speaking, I'm gonna apply that term to ML ops for machine learning or LLM ops for, Gen AI models. And then there's agent ops, this emerging notion of, optimizing the life cycle of building, deploying, and changing agents. Each case, you're doing you're you're preparing an element, whether it's a data pipeline, a model, or an agent. You're building it, and then you're putting it into production, orchestrating, and adjusting it. And I think it's critical for companies to think about this in a holistic fashion. How do you get these teams to work together? How do you get them to help one another? If something happens, from a governance perspective that goes wrong at the agent level, how do you tie back to the model contribution to that and the data contribution to that? And how do you synchronize the resulting adjustment? There are folks that are doing this well. We've got, we've measured we define here at Bark seven different elements of a cross functional AI program that we think is the responsible way and result and the most promising way to approach AI. These have defined AI leaders, human beings that are defining that are are setting the agenda here. Oftentimes, that's the CEO or someone that reports to them. They are addressing security standards and compliance, data access and usage policies, legal considerations, project governance oversight, and and so on. And so you can see that governance is a huge part of this. The folks that do this achieve higher benefits than others, and they also are are more in production with agents. So I encourage folks, if you if you know organizations or peers that are doing these things, keep an eye on them because you can learn from them. JPMorgan Chase, I think, is a great example. They have a number of use cases in in process. Jamie Dimon has said had an interesting comment, which I thought was, something they expect of. Data and AI were too important to leave to the technologist. So he has the head of data and AI reporting to him as well as the president of JPOC because they view it as so strategic. Another company that has made a very bold bet and I think reflects a fair amount of confidence in their approach here is Mastercard. They have, I believe it's called agent pay, which is a an agent to help companies transact business, shop, and so forth. So they're betting their crown jewels on the notion of automating these transactions and agent decisions. So I'll conclude with a few guiding principles here. One, establish cross functional AI governance. You need to address those three domains, and you need to ensure that the right stakeholders are involved from IT, from the, from the analytics teams, the AI teams, and also the business teams. You wanna make sure that you have within IT the developers that are getting involved here. And really understanding that operating model and defining it carefully, there's not enough traction yet. There's not enough run time for companies to say, this is exactly the right operating model for AI and the agentic AI, but there are some, early lessons that we can get. One company that I think is is really good to keep an eye on is Moderna. Moderna has a, a head of, human resources who was promoted to run IT. And the reason she was promoted to run IT is that they view agents are members of a workforce that should work alongside humans. And so that's an interesting risky bold bet, and I'll be very interested to see how it pays plans out, how it pans out. But it's an example of, ways in which companies are rethinking their their operating model. Who does what and how to do it well. You wanna expand the scope of responsible AI. You wanna address new risks and embed ethics in your culture. Training is a huge part of this. Our surveys have found two huge risks, or I should say obstacles to effective data management and support of AI. One of them is data quality. No surprise there. Another is AI skill shortages. And so a lot of companies are investing in upskilling their workers, establishing centers of excellence, hiring talented AI savvy workers, and, we need more of those. And then the final point here is to think about those life cycles and how can you embed the safeguards in each life cycle so that the, the CICD related to a data pipeline is linked to the the feature store that helps train machine learning models and it's linked to the document database that supports a RAG workflow. And then you have the models that derive outputs from those inputs and, and and and prompt agents to take action. You wanna have the whole series of, interdependencies mapped out to make sure those things are all helping each other rather than simply cascading errors. Christian, I will hand over to you. I hope this was helpful. Yeah. It was great. I'm really excited to, move to q and a. Thanks, Kevin. But before we do, you know, I do have to I do have to do a a quick plug for Dataiku because, you know, this topic of governance is so central to how our platform is composed and why we built it. So I've got a few things up on the screen here that that are the kind of area that we plan with regards to governance or small sampling. But really at the core, Dataiku is a a tool that allows both business users and technical teams to work together in one interface to build and create together everything from dashboards to predictive models to AI agents. And, pertinent to this topic in the process, because that's a a unifying layer for that work, we help IT teams to kind of keep visibility and control over all those things in one place, much in the ways we discussed today, during today's webinar. So that would include things like being able to go all the way from creation to deployment of models, deploying it, pretty much wherever, whichever kind of, cloud provider wherever you want to deploy those models and unifying it in one, unifying the monitoring in one place within Dataiku. And we're extending those same capabilities to things like AI agents. So, you know, all ultimately, we wanna have actionable controls, that keep you in control of AI across your organization. So one one unique thing about the way that we tackle this at Dataiku is that we consider our governance to be operationalized, which really, to me, just means that our governance is part of the deployment path. So you have the option to assure ensure that only approved models make it to production, for instance. And it's not just monitoring, but, essentially, you kind of get those actionable controls that will keep you in control. So one additional thing to call out, which I think we'll get to when you go to the q and a as well is, you know, we're working towards also being this single control tower for AI agents as well, which gets pretty tricky, because you need to include in that third party agents and tools and actions that are taken outside the system. So that's, our aspiration in this space and excited to talk a little bit more about that as we get into q and a. Last maybe plug for me is just that still still celebrating that we were recently recognized as the four time leader in the Gartner DSML data science machine learning magic quadrant. And so I just wanted to give a shout out to the team that made that possible, and we're excited to do it the fifth time next year. So with all that said, I just wanted to let you all know that we do have a few ways you can go deeper on this. There's a blog series that Kevin has been working on with us. So this first link on the left here is Kevin's latest blog in that series. And from there, you can find the other ones as well. And you can also if you're a little bit more interested in Dataiku, the link on the right there will take you to our three minute demo. Just get a quick overview of how we enable that creation I was mentioning, and and the governance across all the analytics and AI in your organization. So with that, we'll go to my favorite part, q and a, and I'll I'll quarterback, Kevin. I think I have a few, comments on some of these, but most of these, I think you'll be able to contribute mightily to. So let me scroll through, and I'll start I'll start with this one. Matthew asked us about he says, I see the policy side of this. What about the implementation side? And and all these questions came in while the agent governance slide was up, by the way. So I imagine we're we're talking about agent governance, which makes sense. So he's talking about the nuances of orchestration layer, communications protocols between agents, how folks who are using, like, thousands of agents are doing it. Like, how do you actually implement across those nuances? Curious to see what you have seen and heard, Kevin, on this space of how people are tackling this. Great question. So I think a lot of proven principles apply that have existed in software for a while. I think the the folks that are that are doing this well, are taking advantage of commercial platforms that that pre integrate a lot of the elements and that surface, a lot of the governance information so that you don't need to do you don't need to stitch together a lot of homegrown solutions from the start. So that's one point. Take advantage of automated platforms that are out there. Then another key point, I think, is starting small and looking for a quick win. So start with with where's the business experiencing pain? Get away from the the the science experiment, tendency, which is which is high for all of us, for technical folks. Figure out where there's business pain with an existing process. Where can you improve efficiency, even if it's incremental, potentially using AI, and then work that in that fashion. So customer service is an example of supply chain management. Software coding is, is, probably the most popular and proven use case for, for AgenTek AI so far. And in each of those cases, there are gonna be templates or prebuilt agents that you can put to work, from commercial vendors. And if you can demonstrate some early results, then you can get the confidence and the training and so forth and the budget approval to get more aggressive. So those are some some early thoughts. Take advantage of commercial solutions. Start small. Beautiful. Thank you. We have a another question from Andrew, and I'll I'll take a a first swipe at it. And Kevin, if you wanna add anything, you can. So he asked, what are the best practices you've seen when developing automation, AI agents to execute? So this is maybe more of a builder side question. At least my first thought that goes to my mind is on the builder side, and there's mostly two things that we're seeing, work really well when you're messing with agents and actually trying to get value out of them. One is finding a way to involve business experts because you're probably trying to, either automate or augment some process that they understand deeply that you probably don't. So you we need to find some way to involve them in actually creating and iterating. And that's one of the reasons why, you know, back to Dataiku, why we position ourselves the way we do is we have one interface where those two people can actually work together. Anything you can do in code pretty much, you can do visually, and it's it's just easy for people to go back and forth and say, well, actually, it needs to do that. So the more you can involve the business experts, we find that you get much better results with the agents. And then, I'd say probably the the second thing is keeping those agents, focused on niche tasks. So more and more we're seeing that this instead of having one Swiss army agent, you can have, the coordination of, like, a coordinator agent and then specialized agents that are doing kind of the smallest valuable chunk of of work. That works better. You just get better outputs for the most part from what we've seen when you start dealing with complex workflows. I guess it shouldn't be surprising. That's kinda how humans are organized in organizations as well. Right? And agents seem to do well in that way, as well. So if you've seen some of our previous webinars, and if not, I encourage you to go take a look at some of the use case webinars we've done. We actually break down some some relatively complex use cases. Like, for instance, we did a, anti money laundering AI agent, and it had maybe, like, five sub agents that all had access to their own tools because these were discrete portions of the workflow that somebody who is very familiar with AML would would see and recognize. And so you wanna codify that into these agents that have niche tasks. So those are the two big things that we see lead to success here. Kevin, I don't know if you had other comments on on this one. Yeah. I would, I think those are all great comments. I think, never underestimate the the human element. It's it's the response to the to the questioner. There's, a lot of ways in which if you can select and train the right people, you can really get some some good results. And, and that gets to training, it gets to, having a center of excellence where you can showcase best practices. You can celebrate wins. You can also, educate one another about things that go sideways. So that's that's, I think, critical, because ultimately, it's the people that are going to to make this work and they need to, have the right visibility into the knowledge that their peers are developing. Beautiful. Awesome. I I'll take this because it's an easy one. What size companies do Dataiku for? We typically the people that are best fit for us are typically midsize to large companies just because of the fact that you want the orchestration and you want to have different groups in there together. Typically, smaller companies might have an easier time starting with either open source tools or smaller, kind of point solutions. We do have some, startup packages, that said. So, especially, I think it's not just size. Like, if you are a small company, but you have this this need that you see already for technical and business folks to, like, work together, I'd say you should still probably talk to us because we do have packages that are geared towards towards start ups in in that situation as well. So let me move on to another one. What are, oh, here. Then this is a question for you, Kevin. Under what good governance looks like we had those three pillars across the board. Yep. And, the question is, you know, can there be more? Are there potentially other pillars that maybe we just didn't talk about today? Good question. I think those are the big three. I have a colleague who called them the holy trinity, which I thought was very nice. There are definitely other pillars. It would depend on the the company, the use case, and so forth. The three that I have are really more technically oriented. Now, there are there are other dimensions that you need to that I think you could personally bake into those technical domains. But there's obviously there's brand reputation, there's, there's financial performance and things like that, that, some might view as as additional pillars that need to need to be considered. The financial one is interesting because I think that part of governance really should get to governing costs. And, a related part of that is FinOps, this discipline emerged in the last five, ten years to govern cloud costs, to predict, measure, and optimize cloud related costs. And now responsible organizations are starting to apply that to inference costs, To the token cost, how much does it cost to process a unit of data to support AI? And so, organizations that are starting to really scale up their production usage of AI are keeping a close eye on those costs. Because even though the costs are coming down fast, faster than PC memory or electricity ever did back with earlier innovation waves, their usage is going up, and you don't wanna have the surprising monthly bill at the end, similar to a cloud compute bill from AWS. Yes. Excellent. Let me I'll take, the first swipe of this next one. The question is, is there a paradigm where there has to be a supervisor agent to ensure agents behave? And I'll from our side, I'll say that we see this paradigm, yes, indeed. We have it kinda built in to some extent, in some of our offerings where you have these LLM as a judge techniques. You can have either an agent or an LLM that's actually part of the evaluation process for the for the outputs. That's a little bit easier with, like, a chatbot implementation than it is with an agent because, obviously, agents extend into actions, not just responses. So evaluating the actions becomes a little trickier. I I do think it's, still possible and still a good use of LMS to apply them to looking at traces and seeing what what agents are doing and having those compared to evaluation metrics that you will set up. So evals become pretty pretty important with agents. So, the my main response here is, like, yes. I think using agent supervisors to help here is useful, especially because that's how we see, agents executing well. You kinda have a supervisor or orchestration agent or LLM that will then kind of dole out tasks and communicate with other sub agents, and that works pretty well. I think with control, it seems to be promising as well, but, these are these are early days. And, Kevin, I'm interested if you have, kind of input on this one. Yeah. I agree. Agents can help. Evaluator models can help. As I mentioned earlier, we're using natural language processing or machine learning to gauge sentiment or potentially identify toxic content. Those are ways in which you can have, models and agents wrapped around them that help. Humans really should be in the loop, in my view, for the foreseeable future for the higher risk stuff. And one way to to mitigate risk overall is to start with internal users. Internal employees whose judgment you trust because you hired them and managed them. If they can be the gatekeeper for the outputs of agents, before anything gets into the wild, that's a good way to mitigate the risk to organizational brand and reputation and customer relationships. Awesome. Totally agree with that. This is an interesting question. Basically asking around, you know, are there any standards that, you know, with the usage of compound systems, you have all these different elements in the systems. Are there any standards that allow or disallow going full agentic? And I I suppose what this means is can we say in certain use cases or situations that you wouldn't just take agentic steps, whereas in other cases, you might. And I don't know if this is maybe a a standard or a discussion that you've seen, evaluated, Kevin. It's a good question. Well, the engineering answer is probably the right one, which is it depends. And it'll depend on the the use case, the risk level, the maturity of the organization, the technology, the people in the process that support all this stuff. And then ultimately, it'll be a judgment call on that. Yeah, I think it should be a judgment call with the human leader who's who's maybe a business leader with the advice of IT and, and the AI leader. But, I don't think there are hard and fast rules about where not to use agents right now. The I I would sort of flip the question and say, there are no hard and fast rules about where you should use this stuff either. Start with do no harm. Because there there was this fear of missing out, as I mentioned earlier. Now there's a a rightful fear of messing up. And, you may perceive your peers as rocketing ahead, but they're wrestling with risks too. And I think it it erring on the side of caution is important and taking a close look at the risks, including the risks you might really not have thought of before. You know, say to yourself, what am I missing? What are the other things that could go wrong before you move into production? I think that's a great segue into, this next question, which I really, really like. Casey asked us, you know, can we really have continuous oversight of agents as their autonomy and interactivity improve? So already in the cybersecurity field, people are suffering from burnout and properly conducting oversight. So how do we think about, you know, we're adding this whole new agentic burden, potentially? Just asking for our take. What what is your take on that, Kevin? Yeah. So alert fatigue is has been a an established problem for a while with security and with governance in general. And that would be compounded if you have multiples, a semi or a fully autonomous agents talking to one another. And that's where metadata becomes really important. How do you abstract from all the the telemetry indicators, the sentiment indicators, and so forth about what these what these, agents or models are doing? And, I I think start small, start slow. I I I I don't have an easy answer to that, but I think starting small and starting slow is absolutely critical. Right. Yeah. And I'll just I'll just weigh in there as well. It's like that's that's some of the intent and purpose behind some of the things that we've been releasing in last couple of releases at Dataiku as well, and we'll we'll continue on this front, evolving governance for the agentic era, to the extent that you can. You wanna have things in in a single place. And right now, that's don't not really possible with agents, especially with the interactivity across, different, like MCP tools and and agents that are talking from one system to another. That is, like I mentioned before, that's one of the aspirations. I think the Dataiku is building towards as much as we can having a central control tower for that is gonna be super critical. And being able to apply things that will help us, like, can we apply LM as a judge techniques along with evals to at least make the burden a little bit lighter on the humans that need that probably still need to be in the loop in the near term. There's probably not an easy answer to that right now, but we definitely can't just if you've seen the memes about, senior engineers, senior software engineers that are having to deal with the AI generated code from their juniors, We don't wanna do that to our, like, CIOs and our security people, people in the IT admins and our and our CSOs and stuff where we have to, like, now give them a whole bunch of a slop to go through on top of everything else they're doing. So as much as we can minimize that, it's better. So to Kevin's point, I think the reason I wanted to go to this question after his answer was, like, you probably wanna make sure that you're using high value use cases then so you're not just experimenting and burning people out for for low value. Starting discreet, starting small, and learning the patterns is probably super helpful. Yeah. Yeah. Okay. Let's go to another one. So there was this question, by Hugo that says, you know, how do you see in highly regulated industries the fact that continuous regular regulation will demand governance operations that go beyond the enterprise's doors? Maybe some of what we're talking about here. There's these models, provider agreements, for instance. And alongside, data residency and agent monitoring, all these areas that create this overburden in in costs for CIOs. Any any tips there as we go beyond just what the highly regulated folks have within their own, wheelhouse? That's a good question. So the question is how to how to mitigate risk and what are the governance best practices if if compliance means accounting for what your your partners are doing with your data too Yep. With your data and so forth. That's a good tough question, and it will depend. I think that, it it really helps to start with internal users because then you're introducing fewer risks into your exchanges with external parties, which which is important. So I would encourage the folks that are highly regulated and their interactions with others are highly regulated to look at those internal user use cases first. And only get into the external ones once you have a high level confidence about what the risks really are and your ability to mitigate them. So for example, you might have, legal documents that you're that you're drafting using a, small language model that's been fine tuned on on legalese in your sector and then has a rag workflow underneath that, provides internal documents to help draft that stuff. But you still need an internal trained human attorney to approve it before it goes external. Now the language model is still going to improve and the agentic workflow is going to improve the productivity. It's gonna speed up how fast you get that contract done, but ultimately, it's a human who's gonna be the gate before it goes outside. And I think that would help satisfy, regulatory concerns. One additional thing I'll point out, it's somewhat related is that, AI generated content is not, cannot be trademarked. And so organizations need to think hard about making sure they're not putting something out into the wild because they don't own that intellectual property if humans didn't create it or at least have a very high role in in, creating it. Interesting call out. Might have time for one or two more here. So let me ask you this one. Just to get out of the technical weeds a little bit. And Rob is asking us, you know, from a tactical perspective, the agent model governance were well described here. So good job there, Kevin. What are you seeing though at the strategic and board levels around governance for AI? I think that there's a as I said before, there's a rising recognition that that this stuff matters. And so in a sense, this could be a time of real renaissance for governance teams because they're finally gonna get the resources and the attention that they need. There is a desire to, demonstrate business results at acceptable risk levels, and there's a frustration among a lot of executives. They're not they're not quite getting the results yet. I think most executives continue to believe that this is transformational and that, there's a once in generation opportunity here to improve productivity and accelerate in incremental bits since in coming years. But there's also a desire to improve governance. And so at the board level, I think that board members are asking some hard questions about what this will do to the organization's brand and reputation, which ultimately boils down on to human to human, relationships. And so they're gonna want proof that when you put a model at work with customer service, with supply chain management, or other things, it's not going to poison those relationships that are really the crown jewels of an organization. And so that that figure I provided about the from the global risk advisory board that, AI misuse is the number one reputational risk. Ahead of Elon Musk, ahead of, ahead of opioid association with that, all these things, that's getting board attention, and they want to make sure that you're not taking too many risks. Awesome. Let's do one, maybe two more. So we got this question. How much of, this governance is responsibility of the platforms that provide the agentic frameworks like cloud providers or third parties? And I'll take the first swipe and Kevin hinged to hear what you say. But it's interesting because right now, it's definitely kind of like we're gonna give you the tools and then, much to, like, what OpenAI is saying even as they discover risks, it's kinda like, good luck to you. Make sure that you do the right thing. And there may be tools in place to do things like, view what the agent has done to try to take away some of this black box situation, but there's not a lot out there with regards to, like, plug and play. This is fully governed in the way that you need it to be day one. And that's, again, one of the reasons why we have kind of, Dataiku. We've built our platform. What we have is that the things that you do create are automatically, governed. And, just like you would with ML models and things like this, you have these registries where things are together, and you can have certain workflows and approvals applied to those. But there is still this emerging level, I think, of understanding what it actually means to have, like, central visibility and governance and control of agents that go across systems and talk to other agents in third party systems. That's still, I would say, like, leading edge that that we're figuring out and working towards. And so no platform today, I think is gonna have the the clean and ready answer for you with regards to that. But the best you can have right now, I think, is tools that will allow you to at least get the visibility and a path to auditability on what the agents are actually doing across the organization. And then I think we'll we'll continue, you know, increasing our capabilities there as we go forward in time. Kevin, maybe you have a different, input on that. No. I agree. I think that, we're moving very fast. Lawmakers are, in The US at least, loathe to regulate this. And as a result, the onus for governance very much falls on the adopter. We haven't had a moment like the creation of the FDA where suddenly you could have high confidence in the quality of the food you're eating. Everyone knows that it's buyer beware. You know, when you look at Grok, and and it going rogue with its, with its nonsense and and racial epithets and so forth. When you look at the the travails of Chatt g p t with more factual hallucinations, the the all that demonstrates that you're on your own when you put this stuff into into your environment. And so the more you can take advantage of governed frameworks from commercial providers whose value is stated to be governance, the better. But ultimately, it's it's on you, the adopter. And I guess that makes things interesting. Yeah. I think maybe that that makes me wanna go more comment is, like, even just with traditional AI governance, ML governance, we see with a lot of our customers that they're like, okay. I know I need to govern it and control it, but, like, what does that actually mean? And so a lot of times, it can be helpful even just to have templates and frameworks that are kind of ready made as a starting point for you to apply. So if you're shopping for this kind of thing, I would say look look for that especially because this is kind of an emerging area and at least the people who are thinking about it all day might have some frameworks to get you started. And another plug for Dataiku, that's something that we do is kind of have those templates and frameworks we call blueprints ready to kind of go to apply on top of, on top of what you're creating for that very purpose. Like, all of us are kind of learning as we go with this, so we wanna make it easier if we can. Definitely. All that said, I think we're we're wrapping up. We're out of time now. I appreciate all the questions. We got to almost all of them. Thanks, everybody, for your high engagement involvement. And, Kevin, thank you again for joining us today with an awesome session. Hey. I really enjoyed it. Thank you. I encourage everyone to, engage with me on LinkedIn, follow. We provide, practical data and AI perspectives, and I always like to have dialogues with folks like you. Awesome. Thanks, everybody. Have a good one. Okay. Thanks.