Video: Design AI Agents You Can Trust at Enterprise Scale | Duration: 2764s | Summary: Design AI Agents You Can Trust at Enterprise Scale | Chapters: Introduction to Dataiku (2.96s), Orchestration Layer Importance (97.585s), Structured Visual Agents (192.915s), Structured Visual Agents (324.9s), SVA Use Cases (564.335s), SVA in Insurance Claims (847.565s), Agent Deployment and Governance (1113.66s), Closing Remarks (2760.028s)
Transcript for "Design AI Agents You Can Trust at Enterprise Scale": Alright. Hello, everyone. Thanks so much for joining. I'm really excited about this one. So just some housekeeping things. If you've got questions, throughout the webinar, please throw them in chat or in the q and a session. We'll pick those up and get to them, at the end. We have some dedicated time for that. The session is being recorded. And so after the session, you'll receive the recording, over email, as well. So introducing myself, my name is Chris. I lead our team of GenAI field specialists here at Dataiku, and I'm joined by Youssef, Hi, who's. in my team. Cool. So couple of slides here. Just kind of introducing Dataiku and what we do. So Dataiku is a platform for AI success. And from the start of of when the when the company was created back in 2013, we had these three fundamental pillars that we always kind of believed in, and that's our core philosophy when we built the platform. Those three pillars are people. So regardless of kind of your technical skill sets, your level of expertise in technology, you should be able to contribute to AI success within your organization. Next is orchestration. So regardless of where your technology currently sits, which technology you're using, you should be able to leverage that and connect to that to get to that final outcome. And then, of course, governance. Right? So governing and making sure there's no blind spots across your organization, having transparency in terms of, like, what are peep what what people are doing with data, what the models are predicting, and what are the outcomes that they're driving. Now that being said, diving a little bit deeper into the platform itself. So we can see that the orchestration layer is is really, really key. Right? That is what what fundamentally is making everything work on the. So you can see Dataiku as an orchestration platform that sits on top of your existing infrastructure. So whether that's on prem infrastructure, cloud infrastructure, some organizations that I work with have both or sometimes three different forms of infrastructure that they have legacy technology, future technology. And then we sit across the entire kind of agentic and analytics pipeline. Right? From kind of traditional data preparation and modeling all the way through to building out those agents, connecting those agents to those traditional datasets, to that unstructured data, to those time series models that you're creating, and again, regardless of your technical expertise and skill set. Now this becomes increasingly important, especially if we're thinking about kind of agentic AI. And so the way I think about agentic AI, and fundamentally what it is, it's it's really complex systems. It's kind of in the nature of agents. Right? Agents need to take actions within your organization. And so they need to be interconnected to your infrastructure, to your people, and they need to be governed because the nature of agents is that that they're not deterministic. So we need to be able to understand exactly how our agents are making decisions. And so by having a platform that sits across your infrastructure that everybody in your organization can use, You're essentially driving three key things. You're accelerating growth, you're improving efficiency, and you're building that resilience into those critical systems that are essentially helping you make business decisions. Now for today, I'm really excited because we're gonna discuss some really cool new parts of the platform, namely structured visual agents. So that's the first thing that we'll kind of go in go into here. We'll discuss why why they're important. Then we'll we'll kind of discuss them what what it looks like in practice. When would you choose a structural visual agent over a traditional React based agent? We'll dive in into a demo and actually see it live in practice. And then finally, as I promised, we'll have a q and a at the end as well. So please, as we go throughout the session, do leave questions. We had some really good questions in the last session, so so I'm I'm expecting good ones in this one too. So with that, Youssef, over. to you. Thank you, Chris. Yeah. Just I hope this was a really clear presentation from Chris about the broad AI infrastructure offered by Dataiku. And from now on, we will focus on structured visual agents, which is a new feature we have shipped within our latest release at Dataiku. We will see how it looks like in practice, then the exciting part of the demo showcasing it. And, yeah, just as Chris said, we'll be happy to answer all all your questions, so do not hesitate. We'll have a dedicate session dedicated session for it at the end. So first question would be why structured visual agents? First, understanding the pain point of where that is that this does this new feature come from, and how are we transitioning from probabilistic to deterministic AI. So the the user stories we are covering today is of two people and two personas building agents on Dataiku. One which is a code of data scientist and one which is a bit less technical and a data scientist building agents visually on the table. Both are sharing the same aim and target, and what they want is to chip quick, secure, governed, and performance agents to non expert developers and end users. Historically, the types the two ways people were able to create agents in Dataiku was through one simple visual agent. If you are already familiar with with this feature, it is basically a core React loop, which at at its core has an LLM embedded with some tools to take some action. And for more technical people, we had customized customizable code agents, which are fully customizable through code and writing codes through popular frameworks like in order to to have the intended behavior. And the main pain points on both personas for the first one for visual agents was unpredictability. As as you can guess, having a simple call loop with tools, we could not enforce business logic for high stake industries like like, for example, insurance, which is the use case we will be covering today because we would be requesting past the agent or asking nicely the agent to follow some some pass through the prompt, but never it's never guaranteed that it would be the case. And second pain point was complexity for code agents as this comes up with all the software engineering complexity needed in order to develop them as we should we should develop all the orchestration parts handling the the looping, the execution, and all stuff, which is way beyond the functional agent coding. So that's why we have developed structured visual agents or as we call it internally, SVA. And SVA, you can think of SVA, the right trade off between probabilistic AI and deterministic AI. So I would define it as a set of blocks which follow follow one the other, and it could be either some LLM and general based blocks or blocks related to pure logic, which makes it possible for for the AI to follow some deterministic paths in order to reduce operational risk. It's also a white box as transparency is guaranteed here, and we visually, we could see the different branches and branching and routing the agent would follow for for a given task. And, also, it's reduced cost and latency because because we are we are we kind of removing some of the work needed, and that was done by the LLM in a core react loop as we are embedding the logic in some deterministic way here. So at the high level, this is what it might look like. So just, yeah, to to sum it up very quickly, it could be we could have some LLM or Chen AI blocks, as I mentioned, which are here. We could have routing calls to machine learning models to to tools to whatever blocks we have either built in in Dataiku or this is customizable and extensible to through plugins development. So if you need some proper logic, you could do it either through Python blocks or through plugin development in Dataiku. We could also enforce human in the loop for very high stake decision making. The agent would need human approval before validating that. So this is both for approval or edits by human, and we will see this in the demos later. So that's that's the key the key intent of SVA to kind of ally both both of the best words in terms of workflow and react agentic loops. And let's now see some complete examples of how how some some use cases could leverage SVA in order to gain latency, quality, and also cost. So first first natural use case is predefined workflow and branching. So as I was saying, if you have different branching, we could not we could hit the wall of unpredictability just by asking the LLM in the prompt to do it. So this is this is tackled by SVA, and the the sequence is enforced by the graph. Second is, for example, if we are expecting some some inputs from the user way up front, historically, we would have done it in in React loop as a just asking the LLM to to to to trigger a question to the user and hope hope it follows that guideline. But with SVA, we can have explicit exit conditions very, very quickly within the the flow so we can iterate faster and have faster turns with the with the user. And last but not least, also for all the orchestration stuff, like parallel investigation without all the overhead of writing code. If, for example, we are exploring some independent knowledge banks about independent anti documents, we do not need to do it sequentially as React loop would have done it, but we could run it entirely within SDA. So the main family of blocks, I like I like to think of this in terms of color. So everything that is pink is related to GNI. We could have a core loop, which is the the old visual agents, which is still available for simpler use cases and quicker prototyping. We have also some other GNI blocks, like simple LLM request, delegation to another agent for multi agent orchestration, two calls either through an LLM or directly through within the flow. We have some orchestration blocks like routing, looping, parallel execution, reflection over a turn. Then we have memory blocks. So I like to think of it of this in terms of short term, midterm, and long term. So scratch pad entries would be our short term memory, which is within the current term we are having with the agent. So if we are inputting one ID and we want to remember it for the current turn, this would be a scratch pad entry. Then state entry would be for the full conversation we're having with the agent. So if I'm having a follow-up question, I do not want to to set up the ID of the claim I am asking about again, and this will be a state entry. And then we will see later more long term memory blocks that were customized and plug in developed, which are more about the implicit feedback the agent gathers and this self enhancement loop the agent can can leverage in order to to better its responses. So I'm trying to move to the slide after this. I don't know, Chris, if you can if you have control, you can get to the slide just later, or we'll move to the demo directly if it's not the case. Is this the site after this one? The user interface one? Yeah. I'm not seeing it on my screen. I think we're on the user interface, now. yeah. It works. It was stuck on on my end. Is it good? Okay. Cool. So, yeah, quick quick overview of the UI. I will be very quick about about this slide because we will see it in the demo. But, basically, you have you have to define your blocks. You have to define the starting block. Then this is basically chaining block after the other, defining the next block at each end of iteration of a given block. So how we are we seeing SVA in action today? Today, we will tackle claims insurance use case because this is a very high stake domain, which requires a lot of reading of struct both structured and unstructured documents and reviewing cross referencing policy rules and drafting responses by hand, which is slow, hard to audit, and inconsistent. And we thought that SVA would be a very cool way to replace this strange workflow. So, basically, we will have some input, which is an insurance claim by a customer, which might be, like, a water dam a water damage or fire damage or theft. We'll have the supporting documents of the claim, which would be, for example, for for theft police report supporting photos. We'd have our policy documents within our insurance field, and then the agent will would pull up all this data in order to investigate the right decision to take, which might be approval of the claimers ask or requesting more info if something is missing or if we think that this is not compliant to our policy, ask for some escalation to a to a manager reviewer. So I will yeah. So moving to the demo. Just looking to share my screen. So I think screen share disabled from my side. Is it on your end also, Chris? Yeah. Should be good. they there, here we go. go. Yeah. Because we don't have the slides anymore. Okay. So before I run a bit anything, I will just do a quick quick overview of of the flow we are having. So for those of you who are not familiar with Dataiku, this is the main entry point of Dataiku, which is the flow section. And, basically, you can think of this as all all the way your data is flowing from ingestion to preprocessing to building both ML algorithms and agents and also serving to to end users. I like to think of these blocks in terms of color also, so it's consistent with the colors I was showing. So blue would be for data. Pink would be for. And, of course, this is fully agnostic. Dataiku is agnostic as a platform, so this data does doesn't have to live on on Dataiku. This is I'm pulling this from one e c two server I'm I'm running, but this could be from from Snowflake, Databricks, or whatever. And same on the LLM providers and models. This is fully agnostic and switchable. So if we're looking at the data I I was presenting, we're having three three scenarios, one of water damage, one of theft, and one of fire damage with different different coverage limits. And we are having the claim documents which are supporting these these damages. For example, you can see for the water damage, some supporting photos, the report, the report, the homeowner statement. And then we have our knowledge bank, which you could see here, which is feeding our agents, and our knowledge bank is is basically the policy of our company about different types of damages. And so we can now look at the flow we are covering. This is this this is the UI I was showing in the slide about how SDAs are built in the. So I could ask it while while it runs. We can review the flow together. Sorry. Okay. I will ask you to create the claim with the ID. I chose one of the three claims we we were seeing within the structured data. And while it runs, I can show you what what is done by the agent. So, this is what the flow looks like. And as you see, this is visually rich, and I think this is really a cool feature to to be able to explain what's going on without any black box or or hidden logic, under the hood. So first, we are collecting the claim ID through an using GPT five mini. And I think one key thing to note here is that you could use different LLMs based on on the type of tax task and complexity. So for collecting a simple claim ID from the user request, it might be not that hard. So we're using a cheap and fast model while doing some assessment, and and high stake decision would be more complex and needs reasoning models, we would be using GPT five, which is larger. We are collecting the claim ID, and based on the format of the claim ID, if it's correct or not or not, either we are telling the user that this is not the right format we are expecting and we are getting out of the of the chain directly. Or the other way, if it's good, we can start our functional logic first looking doing lookup of all the structured data in order to to get the claim all the claim details I was showing within the table and set them as state entries. So for follow-up terms with the agent, I I I shouldn't care about telling again about the claim ID or other other stuff or description. Then we have two parallel investigations that run at the same time. First one is a reg lookup, which is a dynamic lookup over our insurance policy knowledge bank. So this is this is to make sure that the claim is compliant. The amount of the claim is compliant, for example, against our company policy. And on the other hand, what we will do is we will look look up within the folder for all the claim documents. And for each document, try to analyze it. And we can see a key a key strength of SVA here as we we could have done this through a rag lookup as we've done it for the insurance policy. But but we think that document analysis is very high stakes for this use case, and each document should be treated and investigated thoroughly and deeply at its own. So that's why we are we're running a for each loop, and we're making sure that the LLM is not forgetting the detail. So if you think of this within a core react loop, it would get pretty messy and complex if the context explodes. So that's why we're having a a loop over here. And then we have a final assessor, which is gathering all the findings from both the investigations and looping with the user as we will see, asking the user for for approval. So this has run, and you could see that the before logging the claim decision, the agent has asked me if this is the right this is the right thing to to to log. So we can see that it has approved the claim with the rationale behind saying that the coverage is within the 10 k limits, and all the documents are provided. So if I'm okay with this, I can accept it. And then what will happen is that based on that, we will have a routing and an approval letter written to the user based on the routing decision we have made. Key thing to to note here that the approval letter, we know that this is really important for some industries, should be is a structured block with placeholders filled by the LLM. So this is not, like, free text written by the LLM. And so if you have some formatting or wording or tone, which is very, specific to your use case or company, this this is fully doable through through SDA. And you could see at the end that the claim was approved. And if I look at the approval letter, this is exactly what I was expecting with the agent filling the assessment and the rationale part. So the the that's it for for the agent running. I haven't covered these two blocks yet. So we have an in and out blocks that we would be sure that will be run each time. For example, context compression is a very popular one within most agentic frameworks today. So this is run outside of of the core flow run, but it's run every time. We're having context compression, and, also, we're having implicit feedback, which is retrieved before before the run and also stored for self improvement of the agent after the run. And, of course, we can look at this. We have full traceability of of the run. I will share this. So this is Trace Explorer, which is our solution for traces, and you could see that we have the full the full trace of of the path that the agent has looked for this one. And if we look at feedback, you could see that I have, yeah, I have I have a feedback dataset that the agent would be implementing and feeding over and over the runs with the user. For example, this is a constraint to not be overly verbose within the response, and the agent will take that into account within its its next runs. And, also, the claim decision log, which is the one I validated now, it it took the human approval before writing to to the dataset. So we were interfacing, and we ran the SVA through our SVA panel, but this is more for building. The next step would be to do to run some evaluation on our agent. And this is this is doable through our also within our newest release, which is agent reviews, which offers you the opportunity to to test and iterate very quickly on your on your SVA build. So as I was saying, for this demo, I have prepared three scenarios. So one three with three with different different reference answers and scenario pass that would have been taking. For example, for this claim, we are expecting an escalation while for for this one, we are expecting an approval. And then we can see that this is we could run this three the three agent runs against the ground truth and then compare the results. And you can see that this succeeded, and even we can compare all the old runs against the newest ones. And this is really important to see the iterative process of how the agent behaves and how it gets better. For example, this is really important to know which LLM to choose. So that's why that's how I've set only GPT five for the most complex task and kept more small models for the other ones because it was sufficient to get the full evaluation done. And I was saying we were testing through the SDA panel. Another way to test it is through agent hub, which is which is our go to solution at the Daiquo in order to serve enterprise agents and to end users. And this is a more nice UI and a nice way to to go down and and run both together and to interact with agents within your project and instance. And I have the same agent linked here, and you could see that we have the same level of detail running it, seeing all the the the the path the path took that the agent took, all the blocks that it has run, and even the human in the loop went inside. And, also, at the end, we could see the final the final assessment validation by the agent. So that's it for the demo. I'm very excited to answer your questions with Christian, and I hope it was clear about how we we could unlock both the deterministic I would say, the deterministic way SVA brings to the old and probabilistic key strengths of core loops. Yeah. And I think if if I if I kind of look at the organizations that we're working with at the moment, I think that everybody is is very much convinced by the power of of agents. Right? They're they're they're an amazing tool, but they are just that. Right? A tool. They're not the be all end all. And so I think, as you just said, structured visual agents bring bring a way in which you can really define your workflows in a more deterministic manner while still leveraging that tool of the LLM and of the agent to do certain tasks within that workflow versus just relying on the LLM and hoping that it, you know, it does your instructions as you've defined them. And if you can set the instructions, they're actually logical operations instead of just just prompts. So really powerful. We've already seen some really powerful use cases being built, some some very complex use cases being built with, as Youssef showed, right, parallel execution, longer term memory as well, four loops, all of these all of these concepts just allow you to go that much further when building when building out these these agents. So we've got a lot of questions here, and so we can kind of have a have a discussion around this, Youssef. I'll I'll bring some here. The first one, which which I will answer here from Mike, is who governs the agents? And it's it's a really important question. So as you saw, right, Dataiku makes it really easy to understand not just the the agentic side, but, like, the entire pipeline that we've we've built out. In this case, it was actually a relatively small pipeline, but but real use cases go go much further than that. And so we're bringing everyone along that journey. Right? We're not forcing people to be able to understand lines and lines of code to be able to kind of understand the intent of of the use case and and of the agent. And everything that you saw was within the design environment. So this is where we're developing these use cases. Now we have what we call our governed environment, and the governed environment is where we can monitor all of the performance metrics and the reviews of all of the agents, data quality, machine learning models within any given project. And any given project might have that. Right? It might have unstructured data. It might have normal datasets across your organization that have different data quality metrics that you need to monitor, and then, of course, agent performance metrics that you need to monitor, as well as intent of the project and business value. And all of that requires different people to sign off. And so the govern part of Dataiku, which is really, really powerful, allows you to bring all of these different people along the journey to see what is the project actually doing, what is the intent, and sign off on their specific things. So you might have, you know, the data science team or the center of excellence signing off on the agentic performance. You might have the data engineering team or data quality team say, That makes sense from a data data quality standpoint. We we sign off on that pipeline. You might have IT or architecture sign off on kind of the deployment pattern that you're going to use. Maybe we're gonna decide to deploy this, you know, on top of our Hadoop ecosystem or on top of our Kubernetes environment. So, again, we're bringing in all of these people because I think fundamentally that that is the most challenging part. Right? Bringing in all of these people, all of these different systems into one single place so that you can effectively and efficiently build these these complex genetic pipelines and more importantly get them in get them into production. So there's some really cool videos which we can share as well on on Dataiku Gov. It's a really, really powerful part of the platform and very flexible as well, you can you can really build this out. Another question here is is an is an easier one. So this one is in which version are agents available in in Dataiku. So the normal agents were available in thirteen point four, which was quite a while ago that we released, I think, almost a year ago now. And then structured visual agents are part of our most recent release, so 14.4, where you can you can find that. Youssef, maybe one for you here. So data queue agents, are there any state variables available to agents to save intermediate data and. deliverables? I think this was a key key aspect of the demo I just presented, which the way we try to think about it, like, they go is to have the full spectrum of of state. So as I'm saying, short term would be through Scratchpad, and this is you can think of this as, for example, a way to manipulate structured data within your flow. This was this was the case for me building this use case. For example, asking for the claim ID, then I would save it as a as a scratch pad value that I would use later within the flow, within another block to, for example, do the lookup. Then within the spectrum, we will have an an intermediate level, which would be state, and this would be for the mut mutate term aspect of a conversation. So if I'm saving claim ID, I want to keep it for the other terms I'm having with the agent. And then on the long term part, I've showed this through implicit feedback, which is a way the agent improves and tries to learn from past conversations with the user in order to identify constraints or positive or negative feedback or what went on talking to the user. Yeah. Take the next one also. So I think it's covered. It my answer kind of covers the next question also from Ilona. Does does it exist does tracing exist an an automatic analysis of the content to improve the agent agentic system automatically? So this safe improvement loop is doable through these blocks. I've presented the semantic feedback blocks, but we have also long term blocks, which is which are more about the facts. For example, asking an agent to always answer in a given language. This is more long term memory effect than an implicit feedback. And tracing, yeah, tracing is available through Trace Explorer, which is native native solution embedded with the within the agents we are building. Yeah. And then we got some other questions on how does Dataiku compare to, in this case, another solution that I haven't heard of. There's, I think, two or three questions. And to be honest, we get these questions a lot. Right? There's a lot of software out there at the moment specifically helping on organizations building out these agents in either code or no code. My kind of two takes on this. So I'd say that Dataiku is platform where you can build agents, but it's much more than that. Right? It's fundamentally a platform for AI success. And when we say AI success, we really mean that doing everything that you need to do in terms of getting AI into production. And now anybody that's worked with AI will tell you, and I'm sure that that most of you in the in the chat know as well, that building models and building agents is actually the easy part. Right? That's the fun part. We're really happy when we get that. But a majority of the time is spent connecting to data. It's preparing that data, making sure that you've got it in in the right technologies, the right vector stores for for for unstructured data, making sure that you've got the right machine learning models in place that can feed into those genetic systems. And so some of these other solutions that are really focused on agent building can be really good at one specific task, But I believe, and and I we know this because we're working with customers that are deploying lots of these agentic systems into production at scale. Building agents is just one parts. It's it's all of the other stuff that that's the hard stuff that you need to do that you can't let become a bottleneck. Right? If you democratize the building of pure agents, well, what if someone then needs to build a time series model? They might then have to wait for a data science team to build a time series model for them and get that into production to then finally connect into their agentic system. And all of that creates bottlenecks along the way. And so Dataiku is aimed at addressing everything across the entire pipeline from the idea stage. Right? I have an idea for a data project all the way through to getting that data project into production in one single governed and controlled platform for everyone no matter your skill set. So whether you're a data scientist, you can use Dataiku, and you will love some of the advanced functionality that we've got with Code Studios, or whether you're a subject matter expert that is doing their first agentic use case ever. Regardless of that skill set, you can come together, work together, collaborate, and build out those use cases without running into those key bottlenecks that you will inevitably run into if you're just trying to build an agent. So next one, there was some questions around compliancy. So HIPAA compliant, yes. Are HIPAA compliant with Dataiku software as a service. It's HIPAA compliant. I can I can provide that? And then in terms of, like, data that we use to train other AI models, so Dataiku, as we discussed, is an orchestration platform. So we connect into the model providers that you have agreements with already. And so you might have on premise models, which we can connect into whether you're running those with with NVIDIA, or you might have an enterprise agreement with some of the other model providers out there, Azure or Anthropic or OpenAI directly, which we would then connect to. Right? So Datadog itself is not a model provider. We connect into the contracts and the existing technology that you have already killed. We sit on top of that as an orchestration there. One from Chris. Maybe, Youssef, an interesting one for you to to tackle. So how often or in what time cycles will you retrain to avoid data drift and knowledge decay? Okay. Regarding maybe regarding API, I think the the main the main aspect and feature I was leveraging a lot was agent review. As I was saying, so I presented one aspect of how how I I try to to assess the size of the model needed for each block. I'm trying to to to come out to, I would say, Pareto optimal kind of flow state in order to run it, but this is you could think of this for all the other blocks. I don't know. Running a call loop instead of a simple LLM block or having just a two call trying to increase complexity. I would say that you should I think you should leverage the platform to in order to iterate as fast as possible. And this is I think this is the key strength of the is that you're you're able to quickly see the results without without neglecting the part of moving to production later. And so in order to iterate very quickly, I would suggest that you start first by building your ground truth dataset. What are you expecting from your agent? What path should it take, and which path should it take for each input you have? Like, set a bunch of question, answers questions, answers, I would say, best path to follow. And based on that, iterate try to get to the to to to the full test passing. So that would I think that would be the right way to link. Yeah. And I think if the question was pertaining maybe to more traditional models as well that you can build in Dataiku, whether those are regression models, clustering, or time series models, We have that capability fully built in. Right? So an ML ops pipeline that allows you to track over time how a data how a model is is drifting away from the originally trained data, and and it enables you to essentially set up detection automations to to retrain the model if something is detected to be approved to to push into production. So we got like that that whole ML side covered as well. And, of course, those machine learning models will often feed into agents as tools for these these more complex use cases. Again, allowing you to use the right tool for the right job instead of just, like, defaulting to an agent for, like, time series analysis, which an agent just isn't not not good at. Right? It's a language model. It needs a tool to be able to do that. So, yeah, that's all baked into the platform as like. yeah. Louis yeah. Go on, A couple. of very quick answers on practical questions. So will the slides be less? I believe, yes. Just after the the webinar, you will have access to to the slides by email. Also, is the data completely private, or can it be solved to train AI? So as we were both saying, Dateco is agnostic and does not host any of of your LLM providers LLM providers within the within the platform. So you're bringing your own LLM or hosting them locally, but this is fully fully on your site and private end to end. Either on deployment, it could be on prem deployment fully for this. So no. Are there any more tutorials either in the TechWare Academy or knowledge base on SDA? I think this is coming very soon. It's not case yet, but it should be it's a matter of weeks for having it. Is there how do you control costs, especially when evaluating different agents simultaneously? So we have also a solution called the Cost Guard, which is monitoring of costs of. And, also, this is action based. You could define a cost controls at at the connection level in order to set up some cost limits over connections. Yep. This is a this is a good one. So I already have a code agent using line graph. Should I migrate it to structured visual agent? I think it very much depends on kind of the the use case that you have. Obviously, there's gonna be some effort in in in migrating that over to to structured visual agent. So, like, what are the advantages that you you'd see? I think for me, the main one is kind of, like, the ease of maintaining it over time. Right? Over time, this might change. And so it really depends on your skill set. If you're comfortable as a as a coder maintaining that code agent over the next one, two years, then then, of course, you could do that. But you could imagine a world where you have maybe some analysts or subject matter experts that wanna maintain that agent or at least understand what that agent is doing without having to read code. I think structured visual agents are a really good situation where you have that, and you don't wanna be the one always maintaining it, adding new stuff to that that agent and maintaining it over over the long term. So that's probably the the main the main thing that I'd say say there. And then Ram said any audit summary report to log for for AI agents. So everything that is is happening in DataIQ, including, of course, the agents and and their performance is logged. Right? We call these our our audit logs. And so it's actually really easy to to pull those audit logs together to understand what decisions the agent has made over the last three months, over the last year, if you want to, and and use various techniques like LLMs to create a summary or a report of that of that agentic performance. So that would be would be relatively easy to to set up. Let's see here. How can you deploy your agent on your company platform? So this is a this is another good one. So Dataiku is is very flexible in terms of where it's actually deployed. Right? We have three patterns. We have on premise, so we can deploy Dataiku literally on your own hardware, on your own environment, on premise within your organization. We can deploy it in your cloud stack, so AWS, Azure, GCP, and we can also host software as a service. So you can we we host it for you, and you connect to it through securely, and and and you can leverage it that way. And so it really depends on your organization and the architectural decisions that your organization makes. But, fundamentally, we can deploy that agent built in Datacru wherever it's most suitable for you for you as a as a. company. Another interesting one is from. If we have structured data sets, our agents able to identify logic or relation between dimensions. All this requires definitions. So so this is also a cool feature we have at which is semantic model. So for my use case, for example, it was pretty straightforward as I was using only one dataset, so I did not set this up. But we have a full semantic layer solution building dimensions, building relationships, golden queries, also glossary and instructions in order to have a semantic tool that would be able to run SQL queries and use it as a use it as a as is or as a tool within within your SDA? I think we did most of the questions there. Thank you so much. You exceeded my expectations in the quality of the questions. So yeah, thank you. That was really good. As we discussed, you will receive the presentation, of course, over email and the slides as well. So, yeah, I would encourage all of you. We have a free trial on the website. Right? So you can start a free trial if you if you've never tried Dataiku before. I really encourage you to do so. We have academy.dataiku.com. That's a great resource where you can go and learn all about the platform from data preparation to ML ops to agent ops and everything in between. So, yeah, I go encourage you to to try it out, to to play around with it, come up with use cases. Yeah. Thank you all so much for joining, and I hope to see you on the next one. Bye bye. very much. Bye bye. Hi.