Video: The AI Agent Revolution of Self-Service BI | Duration: 3740s | Summary: The AI Agent Revolution of Self-Service BI | Chapters: Welcome and Introduction (7.28s), Speaker Introductions (102.955s), Evolving Analytics Landscape (205.67s), Changing Data Interfaces (572.28s), AI-Powered Business Insights (1246.52s), Scaling AI Governance (1574.965s), AI for Business Outcomes (2076.555s), Monitoring and Governance (2452.04s), Q&A and Conclusion (2963.035s)
Transcript for "The AI Agent Revolution of Self-Service BI":
Alright. Good morning. Good afternoon for people. Jane from I guess if you're east of me. And maybe good evening for for anybody who's joining from from APAC, though it's very late in the morning there. Great to have everybody here today. We'll go ahead and dive right in. I'm sure some more people will filter in, but, you know, let's go ahead and and just get rolling. Really excited to have everybody here, today. So before we jump in, I'll do a quick little bit of housekeeping just so everybody sort of knows what we're gonna go through here, and then I'll introduce the webinar. So the first one is, hey, please use the chat. Feel free to introduce yourself, say where you're joining from, and throw any questions or comments or disagreements, especially if you have disagreements. Throw those in the chat. We'll keep an eye on it and, you know, try to answer some that stuff as it comes up as we're presenting. Or, you know, we have some time reserved at the end for q and a. So, you know, if you throw some some quick questions in there, definitely appreciate it. It'll help us when we get there. There is a q and a panel, so feel free to put stuff in the chat. We can push push those in the Q and A. Or if you actually have a specific and there's something that comes up that you want us to talk to and address, throw those in that Q and A panel. And like I said, we have time at the end for it, and then also we'll sort of even keep an eye on those as we're going. And then finally, we are recording this session. We will send out the recording and the slides for everybody who joined here today. So, you know, keep that in mind as it as it comes up. So alright. Let's go ahead and get rolling here. So quick introduction here. My name is Conor Jensen. I am the global field CEO here at Dataiku. Been here for nearly seven years, came from a customer before that and spent most of my career in the in the in in industry in a couple different industries across, The States in finance and retail and in the military. Excited to come and sort of share both on this topic here around, you know, the evolving nature of BI in the current, in the current place. But I also have with me Cody Brooks, one of our solutions engineers. Cody, why don't you go ahead and introduce yourself? Yep. Hi, everyone. Good to be here. My name is Cody Brooks. I'm a solutions engineer here at Dataiku. So I've been here slightly, less longer than Conor. I've been here for about a year now. Actually, I think my one year was yesterday. Okay. Been in the industry for a few years still before this. So really looking forward, to this webinar and, answering any questions you guys might have. Awesome. Thank you, Cody. Alright. So as a quick sort of introduction and recap, obviously, you're here because you saw the abstract, and so you know what think what we're talking about, but we're sort of all on the same page. You make sure you're at the right webinar here today. What we're talking about is really how we see at this moment and very much in the near future, the evolving role of what business intelligence and sort of traditional analytics looks like as AI agents and, you know, AI tools sort of continue to become more prevalent, more widespread and more capable, frankly. And so how is that? How is that changing? And, you know, we'll go back and forth a little bit. And so I'll talk a little bit about it and sort of what we're seeing and present around some themes on this that we're seeing. And then Cody is gonna demonstrate, via Dataiku within the tool what the current state of some of these sort of evolving things looks like. So without any further ado, got plenty of people joined. Thank you, everybody, for, hopping in and saying hello in the chat. Looks like we've got people from all over. So, very cool. Glad to have you guys here. So the the, like, couple central themes, that we're gonna be going through is this, how does it look to consume? Especially, we're gonna take this a lot from the perspective of not just that end user that sits in the business. So, you know, if you're sort of from the BI team or from the analytics world, that person that you've been building stuff for, marketing analysts, somebody in supply chain, a finance person, whatever that looks like, where for so long it's been we're creating dashboards, we're creating reports, we are building those and sending them to them. Even when we're sending them access to underlying data as well, that often is here is a read version of it and then they're pulling it out and working with it in an Excel or an Alteryx or some sort of other desktop tool. And it's a very fragile system. You know, I'm sure many, many of you out there, if not all of you, have the experience of being in a place where there's proliferation of thousands in most cases and sometimes tens of thousands of dashboards or reports or things that are out there. But when you really dig into it, how much of it is actually consumed, how much of it is actually used? Typically, only a fraction of those are really being used. You know, we can look at a couple of examples of where we've had customers sort of migrate that type of stuff to Dataiku. And the first thing we will try to do is a rationalization of things that are either duplicative, where you have 15 or 20 different dashboards all saying the same thing, although usually slightly differently. Or you just have, again, thousands of dashboards that aren't being used, and we tend to see, like, at least 80% of the things that are out there don't need to be rebuilt, don't need to be migrated because it's just not being used. And so this is what this world that we're living in from a BI perspective today looks like. And we see this moving into new patterns. Right? And so these aren't necessarily all brand new. Right? Like, there are ways that we work with, things like dashboards. But changing, and we'll talk about that, sort of like, you know, both dashboards being standardized and centralized and flexible in a way that they aren't today. And as well as additional things of the sort of on the fly conversational analytics that we're starting to see via the generative AI tools and agents, but there's a lot of, there's a lot of pieces there. Thank you for the comment, Matt. For those who who have joined webinars with me before, this is the first time that I'm clean shaven in, like, six years or something like that. So it's still weird to look at myself on camera. We also have the analytics and AI generation side of things. Like, how are things being created? Not just those end products, not just a I have a query of some data, but how are we actually creating these data products via natural language, various generative AI and agentic tools. And then the ability to sort of do contextualized analysis, which is often what the role of that sort of end user analyst is, is bringing the context, which means not just the raw data, the hard data, you know, your sort of structured SQL esque stuff, but bringing in unstructured data, bringing in, you know, knowledge and everything like that to create not just here is data, but here's the key messages. Here's what this data is actually telling us, not simply here's a chart. That is starting to be able to come be surfaced, you know, autonomously to those in those analysts to be able to sort of create these things. The next idea here is very much there's this is all cool, but, you know, what are the challenges gonna be in here and what's this gonna look like? Some of these, you know, common sort of questions about how this is working is, how do we make sure that everybody's looking at the same information and it's the right information? That's a core problem that we already have today. You know, we talk a lot about and we will talk some more about today, the challenge of, obviously, agentic or generative AI hallucination, things like that. And the challenge is, you know, when we look at the people that are or they're sort of existing, you know, if I have to ask a question of, you know, human analysts today, we already get a lot of especially talking to executives of, well, you know, if I give everybody access to this data, I get 12 different versions of the truth for every single person who touches it. If I ask three different departments for the same number, I'll get three different results because of the slight little differences and things like that. This is, for lack of a better word, like human hallucination because we're all using that same data, but we put our own spins, our own context on it. And that's one of the challenges that we're trying to work through. And this is where we see this evolving is that this is not a question of machine intelligence or human intelligence, but it's really how are those things weaving together to do this, to create us our common references, to make the, outputs trusted. Answer this question of should all employees be able to do this. I mean, I firmly believe in the yes, but within the guardrails and with the observability that you need to have that allows for things to be, auditable, both, again, those automated machine learning decisions as well as then, you know, human decisions. Alright. So let's dig in here. There are five sort of categories here that that we see this or five dimensions, I guess, that we see this changing along. And like I said, I'm gonna go through some of these and talk about them a little more on the abstract side, and Cody then will then demo this out and hopefully make it a little bit more concrete as what we're talking about. So many of the things I talked about in the upfront here were interfaces. How will how are we versus how will we interact with data? I talked before. Right? There's static dashboards. There's reports that are sent off into the ether that maybe gets consumed. Maybe they don't get consumed. Or there's a ton of manual data manipulation in spreadsheets or spreadsheet like tools where I can't get to the answer that I need to via the dashboard that's provided to me. I need to pull that data out and work with it somewhere else, which then, of course, if we're on the data side, becomes a challenge of not knowing who's using the data, how they're using it. If I'm providing data and and it's not the right data, I need to know that that somebody needs to do it. So what will that look like? Hopefully, as we're changing and going forward, a lot of this is really because this idea of this conversational workspace. How can I have access to the appropriate data in a live fashion, structured, unstructured data, modeling and and forecast data, all of these things woven together in a way that I can access them and see them, but work with and request from the data what I need? I think one of the challenges here too that we'll that we'll talk about this a little bit is we don't want this to be a world of, okay, now I have to learn how to craft just the perfect prompt to get the data exactly the way that I want it. Right? In the same way that when I request from a BI team that they create a dashboard for me and it comes to me, sometimes that answers the question. Hopefully, it answers the immediate question that I have. But the flexibility, you know, and that that lack of, flexibility more to the point makes it challenging for me to continue to use that to answer new questions, different questions without having to go back, oh, can you add, you know, a column of data here? Can you add a slicer here? Can you do this, etcetera? That is a huge change that will be coming. So we'll talk a little bit again. Right? So, like, how does this look to change where I don't wanna have to prompt and, you know, have this, like, super fragile I can ask very specific prompt to get a very specific thing that answers my question. I also need to be able to still work with and edit that data, you know, on in a flexible environment to be able to do that. So what can some of these things look like as we're going forward? You know, we do see the access to data being influenced by and powered by generative AI, but not restricted only to generative AI. Right? This you need to have sort of both of these different sets. And so different use cases, different problems, different things we need to have. Sometimes a dashboard is still great. I need to be able to see a trend. I need to be able to see, you know, which units are doing what. You know, we, especially here in the data world, have a lot of experience working with and thinking about things in graphical type formats, in list type formats. And, you know, sometimes I feel like I think in dashboards almost in terms of, like, how do I wanna solve this problem? Alright. Here's the graphs that I would like to see to help me understand this problem. Again, the sometimes the format that we're doing in them is great, but the fragility of how we create those today is the challenge and moving into having generative AI, helping to create those and helping to do those, as long as it's creating editable and flexible assets for us, it will, you know, start to become easier. We also see, of course, the world of agents really making a big impact here in an agent having access to different data sources, to different models, to different aspects of your data to be able to help answer those questions. Some of these can be enterprise agents that have been built on top of curated data and have very specific tools into them, or especially for people who are used to working with data themselves, the ability to create their own agents. I see I see this personally a little bit of a evolution of the idea of the sort of citizen data scientist where for a long time, you know, there was this notion, hypothesis that people would, you know, become more data science y that your average person can learn. Again, this usually started with, oh, I'd like to become a citizen data scientist, and then the answer would be like, cool. Go learn to code in Python, and then come back and we'll help you do that. That is not I don't think that was ever really a, super realistic way to look at it. I also think that the notion of that is a little bit flawed in that, you know, a marketing analyst who learns a little bit of data science and learns how to use these tools is, in many cases, still gonna be a marketing analyst just using better and more powerful tools, but agents are really changing that. And we'll definitely show some of that, say, of how working with agents can really take what you're doing today. It doesn't mean you're suddenly a data person or you're a data scientist or you're gonna spend all of your time in it. It's gonna help you do your job as an analyst of whatever flavor that looks like better. We also have to think about things like and, very frequently, when we are presenting our findings, when we're presenting our data, whether that's internally, whether that's out to a board of directors, whether it's an investor meeting, a team meeting, whatever that looks like, we're not always presenting that data from within the system itself. We need to pull that out and create a presentation, create a PowerPoint, create a story around that. And that's another area where we can start to automatically generate those. Again, not generating them as a static thing where I have to perfectly be able to prompt this out of the gate, but to be able to take, you know, so if I think back in, you know, as in the finance world, creating those sort of monthly, FP and A type reports where I then go and sit down with the, you know, head of a BU to talk through, here's what we're seeing, blah blah blah. I would go through and create these slides, and then there's a lot of, okay, this unit did this, this unit did this, here's what we're seeing, here's the key message, etcetera. How do we create those autonomously or, again, create the first draft of those for it where this isn't a very, like, manual process to create those? The other thing also that on the back end side of it, so this is we're talking about sort of like outputs here, but there's also a change in the interface of how we find data, how we access what's within our systems as things like semantic search with, you know, whether it's graph databases or other sort of semantic layers in between help us find the data that we need in a much easier and more flexible way. Or we have AI assisted data science where we start to, again, create those machine learning, those data science type products without having to go and learn how to code first or get a degree in statistics first. We should, you know, definitely have tools to help us do that, make sure that we are using the data in the right way, but this is how this is and will be changing. We'll show a couple examples of that. The next theme, though, I think that's really tied to that is the time to insight. So a lot of what I've talked about here in this world is sort of answering questions answering business questions via data has been a very slow and manual process. Right? Sometimes, hours or days, I have definitely had those, cases both as a requester of answers and the creator of answers when I was, you know, doing the hands on work more myself of, okay, cool. Here's a random question. Okay. What data do I need to answer that? Where am I going to do this? How do I, you know, pull all of it together? And sometimes that's quick, and sometimes that's my weekend. And sometimes I ask a question and it takes me a week to get that answer back because it's messy data under the hood, etc. How do we speed this up to where this becomes instant or near instant, again, with that sort of explainable and adaptive nature to it because the answer that comes right away is not always the perfect answer. In fact, it's really the perfect answer that first draft that you get. So, the ability to iterate through that and adapt it is really, really important. As we think about, you know, not every problem you need to solve from a data perspective goes through this sort of full lifecycle. You know, sometimes it's one time I'm pulling the data out, I'm preparing it, I'm doing some analytics, and I'm getting an answer. Sometimes I'm pushing things out to where I need to do actual predictive machine learning, and I'm doing model training, I'm doing evaluation of stuff, I have to validate things. Maybe I'm having to deploy something into a production environment that has, you know, a a model or an agent or whatever, and that needs to be then, you know, monitored out in production. Any way you slice it, these different steps, you know, we see big potentials to move faster through these, whether or not you're doing the full sort of life cycle of a production, you know, reports, dashboard, agent, model, etcetera. Or if I'm just answering a question, you know, that my boss asked me this morning that I need to get him an answer to, him or her, I should say, an answer to. Actually, it is her. So my boss asked me to have to give her an answer. So having, you know, these there's different themes in here. One, the first is just how do you consolidate where does that information come from? How do I get another world of having to go to 34 or five different SQL databases or this cloud and that cloud and this old on prem thing to be able to pull that data together through? One, just from an access perspective. Two, with things like cataloging the semantic search, having quicker tools to be able to sort of evaluate that data without having to do it really manually, and having hopefully some guardrails on it to make sure that I'm using the appropriate data and I'm using it in the appropriate way. Then as I start to process it, again, what are the easier, quicker ways to do that? Some of that's AI assisted. Some of that is through, you know, writing some code. Maybe that can be low code depending on my skill. But also having all of this being traced. Right? So it's not, I came up with an answer, and if somebody needs to replicate that answer or ask me questions about it, we are digging through Excel formulas together. Right? How do we quickly be able to look at the lineage, be able to look at all the transformations that everything's happened in doing that? On the training side, you know, there's a lot that can help to speed up and do that machine learning type of stuff, especially by having the right parameterization, the right hardware used to do that. Again, how do we do that where I don't need to move the data into a different platform to be able to do things, change stuff around? And then on the production side, you know, we talked some about that. That actually, when I think about sort of my time in industry, often it took as long to produce, to put new to production as it did to create a model or or an insight, especially things that were persistent or had APIs and and that type of stuff. How do we shorten that? That can also that can often be the fastest part to speed up. And so we'll show a little bit of sort of how that's changing from manual coding and deployment and everything like that. So alright. So I talked a lot theoretically, a couple of things here. I'm gonna stop sharing the slides for a minute. I'm gonna throw it over to Cody who's gonna go through some of what we see these things looking like, you know, here in the the near term in, in Dataiku, and then we'll come back. Awesome. Let me go ahead and share my screen. Hopefully, everyone can see that okay. You're good. Sweet. So in the past, hopefully, this looks exactly like you all have probably been dealing with BI tools for the last decade, decade and a half. Right? Whether it is it was dashboards in Dataiku, Power BI, Tableau, and your BI tool. Things have looked like this. Right? Your line of business users, your consumers are going in here. They're seeing a dashboard, and they're having to do, you know, drill down manually. They're having to go ahead and slice and dice the data, within here. What that creates is multiple dashboards on top of each other. Right? Because every time someone needs a separate view, they need to go to the analyst and the analyst is then building that view, for your line of business users to be able to see. So that is hopefully gonna become a thing of the past. Now once again, dashboards are not going to completely go away forever, but the AI side of things is allowing us to drive deeper and quicker into insights, even faster. So what does this look like in Dataiku? So this is Agent Hub, within Dataiku. So this is gonna be your one stop shop for your agents. As you can see, it's a conversational interface like I'm sure almost everyone in here has used ChatTubt or Gemini or something like that. From here, your end users can go in here and immediately start a conversation. They have the ability to go ahead and draft their own agent very quickly and share within others into the new organization, or they can choose from an enterprise agent that has been created from them, by your analysts within your team governed by your organization, which we'll get to, a little bit later. But, let's say your n minus business user. Right? They've gone and they've looked at the dashboards and everything, but they want actual insights. Right? They want natural language of what's actually happening within their business. They can immediately just kinda select the agent here, and they can even ask it like, what can you do? The agent is actually gonna go ahead and think, and it's basically gonna go ahead and say, here is what I can do, and what I'm available, for, for you to be go ahead and ask some questions. So this agent, as you can see here, can help me explore, summarize, and compare business documents both structured and unstructured. So from here, maybe I wanna go ahead and ask it what are our main drivers of profitability? And I had to practice typing profitability a few times because my typing is not that great. So from here, this financial analyst agent that's been created for me in the background, we can actually see a list of activities that it's performing. It's actually calling on that agent, it's thinking, it's using a few tools that have been tagged to that agent, which is both a structured knowledge bank search and an structured data knowledge bank search. Once it goes in and thinks it's actually going to give me a response. Here we see a direct response immediately in conversational form. It's telling me revenue composition and growth, expense management and operating leverage, pretax margin probability metrics. So I'm getting immediate insights into my data rather than having to interpret the dashboards that have been created for me. I can continue asking questions from here. So I see supporting numbers and a basic summary. I can continue in stories, which is gonna allow me to create, automated PowerPoints with this information that it's allowed to see me, or that's allowed to, I've been allowed to see within the agent hub. Then a really cool thing is going to be the trust that your end user is going to have with this. Am I trusting that this agent is giving me the correct answers? Immediately from here, I can actually see details. And in that same sort of panel where the activities work for me, I can go into the sources and see, okay, I'm getting information from some PDFs here, another PDF. Let's see. Just a bunch of PDFs. I think let's maybe we'll get a PowerPoint or something here in a second. Here we're actually pulling from structured data of an Excel file here. As you can see, we're seeing that the agent had the orchestration on the back end to be able to go in and search all of these documents. I've given it the ability to search and make reasonable, insights into the questions I'm asking from here. On top of this, right, this has been mostly a conversational side of things. I'm asking a question and answer. These agents are gonna be able to allow you to actual perform actual tasks too. So some of that could be running a model in the background. So that financial forecasting model that we saw, maybe we use this agent to actually run a model in the background. We can do things such as fill in Salesforce fields, craft and send emails, and so much more within the agent side itself. Instead of needing to go through a dashboard, create a new dashboard every single time you're wanting to slice and dice the data differently, the era of agents is allowing us to get insights a lot quicker than than we have in the past. So, really, the point is is to radically improve the access speed and depth of the insights that you're wanting to ask. So I'll go ahead and stop sharing there and pass it back over to Conor. Oh, awesome. Thank you very much. Let's go ahead and jump back in. So one of the things that, Cody mentioned there as you're talking about that on this trust side, this is talking about a little bit. It started my my preamble of you know, this is a challenge for trusting data whatever the source is today. Right? Whether that is a I'm the analyst like digging into the data myself, I'm an end user asking somebody to pull it, for me, or I'm a business person asking, you know, business leaders are asking different departments to do that because, one, the provenance of that data, the lineage, it can be sometimes really, really hard to track, especially when you're being served that up via a dashboard that a team has created and you can't see where the data came into to populate that, or you're getting something that's built in Excel or something there, you know, that obviously just starts frequently. Somebody just copy and pasted a bunch of data from a table into Excel and they start working with it there. So understanding even where that data came from into the systems can be really challenging, unless you really have access to those engineering tools and sometimes the skills to be able to do that to look at the lineage and everything that's been created. The, you know, one of the, I guess, sort of promises to make all ten fifteen years ago was being, you know, being able to put data cataloging and lineage tools in place. But they're extremely manual and extremely fragile, and, you know, that hasn't really sort of proven itself out yet. We see more of that, inherent in as you move your data into some of the clouds or some of the, you know, cloud lakehouse warehouse systems. But that's not where all of our data is. And this is, again, a huge challenge here that we see that, you know, when data is not all in AWS or Azure or Databricks, I've got some data, probably in most cases, in AWS, Azure, and Databricks and on prem systems, SQL things, old, you know, sometimes still even like mainframe databases, etcetera. So we need to be able to understand where this data is coming from and how has it been structured. What is that lineage? What is the what are the definitions of this? You know, if I'm talking about a type of revenue, I'm talking about something there, you know, did this get defined the same way both in terms of is this the same structure, but what were the filters that were put on how that data was pulled, etcetera? This is where we see in the new world really moving into a place where the system itself underlying wise has very much shared metrics, very transparent and observable logic with humans as oversight every step of the way and exposed in a way that that human oversight doesn't have to be an engineer or somebody who has those sort of really technical skills. You can bring the business people into that to be able to do that as well. So this is, you know, so I came as I mentioned before, I came out of the financial services world into the tech side. This is a big deal in regulated industries like finance, like health care. You know, we do have areas that are sort of already force forcing this, but to be super honest, I have had more conversations about governance and trust in the last twelve months than in probably the last decade. Prior to that, one of them is the huge question of scaling AI. Right? AI is getting much bigger. Also, oil and gas. Good point, Max. Thanks. You know, there are there are industries where this has already been prevalent, but it tend to to only even be in specific, facets of those industries, or there are functions like HR that, you know, maybe have a little more governance around this. But AI is becoming ubiquitous across our services, across our systems, and our ability to know who's accessing what data, what data is being surfaced to them by the AI, especially is this with the appropriate sort of guardrails on that data, how do we see this. And the regulatory landscape is very, very fluid at the moment. You know, a, many companies, and I I imagine many of the people here today are from companies that are existing in a multinational environment. So it's not just what's happening in your country or in your continent. It's how do I deal with data that I have that's in the EU versus in Japan versus in America, etcetera. So there's a lot that we need to be ready to do when that regulation happens and trying to figure that out as a challenge. Losing trust in AI can happen in minutes is, I think, is a really interesting one. This has very much been a challenge in the data world forever of if you give somebody a dashboard and, you know, they immediately go, okay. Well, here is my here is my last month's report from the finance team and it says that this is our revenue. I look at the dashboard and the dashboard has a different answer. Either I'm no longer trusting or I'm not gonna trust this new dashboard or I'm no longer trusting the finance team. And in most cases, you're actually losing confidence in both. Right? You know, if my finance team says one thing and the dashboard that was built for me says a different thing, Who's right? I don't know, and it's gonna cause, you know, sort of a lot of that distrust, mistrust. As we move into the world of predictive analytics, both from the sort of machine learning side of things and then now into the generative side, that is easier to do. You know, when you get a prediction that doesn't match what you believe or you get a hallucinatory result that you realize is fake, that very quickly erodes trust in that tool that's coming to you. There are certain things that we'll talk about here to how do you put guardrails around that and how to do it. But generative AI is both a catalyst, in making some of these things, easier and more possible to do, but also a catalyst in highlighting the problems that were already there by just the speed and the breadth of which people are working with, AI tools in this space. One of my favorite examples of this, is Macquarie customer has been with us for quite, some time and has, you know, this point in time, hundreds of users both across their back end technical teams and, you know, in their different sort of divisions working with this. And they focused very, very, heavily on AI governance to be able to say, as we're putting these things out, if we don't understand how a decision or if we were to point we can't explain how a decision is being made by an AI, to either an internal customer, to a regulator, or to a customer, this is gonna be really challenging for us to do this. And so they focus very, very carefully on how do we scale the governance along with scaling these AI to tools. This is looking at both, again, who is working with the data. You know, is this someone on a technical team? Is this someone in the business? How do we know that those requirements are being surfaced very clear? How can this be collaborative? Right? So if I, in that old world, so, you know, had produced a dashboard and the person come back and says this doesn't match my thing, how do I, you know, work through that? That becomes even more important. I often, you know, as we're producing data science machine learning type products, would be really frustrated when somebody would say, oh, the, you know, results are bad, so we don't use them. And you hear that six months after you'd, you know, launched a model or something like that versus having that feedback early, helpful, and being collaborative. How do you automate and systematize that collaboration so that it's not, hey. Send me an email if you disagree with the results, but, you know, we put those things into the platforms to be able to do this. So there are definitely ways that we can implement this governance in a way that feels like a lot of, overhead and tax early on, but actually will help us accelerate faster by getting the right foundations there from a governance perspective. It's an interesting paradox, you know, often, especially in the world in these regulated environments, it feels constrained because of, you know, your inability to move as quickly. But when we look at our customers that have sometimes moved the fastest, they're often in regulated industries because those constraints and that requirement for governance has bred good practices that helps them move faster. Our next theme really goes into the idea of, at the end of the day, everything that we're talking about for access to data, access to AI, all of this is about using data to improve a business outcome. At the end of the day, that's what we are trying to do here no matter what, whether you're talking about being a data first company, an AI first company, an AI centric, whatever, you know, there's lots of different terms thrown around over the years. This idea is that we need to be able to use data to help us answer and solve business problems. And in the past, you know, actually, sometimes as I was typing this up and thinking about, you know, like previously, there was a lot of access to data, but inconsistent understanding of it, inconsistent ability to sort of use it. And in a way that would whether it was because of mistrust, because of not access to the right data or the right tools and skill sets to do it, would really mute the impact of those. I remember an example of deploying something, at a company where we were one part of a process, we deployed the tool and, you know, when we went back and talked to those users after a few months of testing to sort of see how it was going, it basically said, well, we have manual levers on the back end after your prediction engine comes in and does it. And, you know, we disagree with it. We then just can use these levers to undo the impact of the model that you have there. And so whether or not our model is right or those users are right is really inconsequential, and, to be fair, probably both of those. Right? Sometimes the model is wrong, sometimes the person is wrong. But if we built out a model that then is not changing and impacting the results from that that business process, then it doesn't really matter. And a lot of this was because of either, you know, we had access to the data, different people had different access to the data, and different levels of understanding, either better understanding of the data or better understanding of the business itself. And so this is really where I come to this idea I talked about earlier of how do we blend what the machines are really good at and the precision and the possibility of the data with the human judgment to make sure that this is a good answer, that this is the right case, this is how we actually want to solve this. So one of the things that is a evolving state here, as we look at things like you know, there's a lot of different themes in here so we can look at, you know, how do we use data for worker augmentation, things like Copilot, to, you know, make it so that I can have a code assist or, you know, an agent to provide me with data faster or something like that. Process automation, using AI to do that rather than sort of hard coding RPA type processes. And taking a set of steps that is maybe even still sort of deterministic, but having, like, intelligence in it to be able to say, here's where, you know, I can move faster, here's where I can skip stuff, or here's where things aren't quite working. But this is still how are we using these AI tools to accelerate the way we already work today? And the challenge and what's coming, you know, in sort of the near future here is how do we stop looking at a process today and saying, Okay, I can drop AI in here and speed up this or I can drop in this, but how do we actually start thinking from the outcome on the way back to be able to change those processes, to be able to change those systems, leveraging AI and data in a different way. This is one where we see really both interesting and and helpful. I get asked on, you know, like, the when I'm talking to industry analysts and stuff like that, you know, Where's the ROI? I mean, there's a lot of money being spent on AI tools. A lot of money being spent on AI tools. Where is the ROI? How is this happening? Obviously, we work with customers in a lot of different places, and we hear from them. You know, I can only share so so much, but to say, where is this value coming from in terms of what are the impacts and the changes, and what's the actual you know, the real at the end of the day, CFO wants to know, you know, I spent this. What did I get back for it? There's a huge variety of places that this is coming into place. You know, I just have a handful of them on here around faster development of things like patents, or faster customer service, faster marketing insights by being able to do things at scale with more, automating things like insurance inquiries, where this is not just a, you know, simple Q and A but needs to be able to sort of dig into a policy or something like that, improving, the speed to market of things, so like marketing campaigns or, the speed of information going to our employees help them make better decisions, not automating those decisions, but giving them more information to do that. There's a lot of different ways in which we're seeing this, some of which are very clearly have real dollar impacts, some of which are, you know, a little bit more subjective and we're still sort of working through to see that. But the underlying thing and and, you know, what I wanna make sure that we think through is that all the data and all the AI doesn't really matter if it's not, at the end of the day, impacting and improving our business. And so that's where we really need to be focused from an outcome perspective. The outcome is not a dashboard. The outcome is not a report or the outcome is not deploying an agent. The outcome is, did that report or dashboard or agent or whatever it was, did that change somebody's decision? Did it get us a better result to that business problem? So I'll pause again. I'll throw it back over to Cody, and he'll actually demonstrate through some of this stuff on the trust side, on the outcome side, you know, how we are how we are seeing that changing. You are muted, Cody. Can we do one thing at a time? I couldn't unmute and share my screen at the same time. Awesome. So, thanks, Conor. So what does all of this look like in Dataiku? Right? So, to start, I think from my role, right, what we're seeing a lot out in the field, is organizations want these AI insights, but really what they actually need is transparency, oversight, and insurance of the quality. Right? So Dataiku is gonna provide you really with the traceability, the monitoring, and the governance, both for the data and the agents and the AI that you're using, both from an analyst's perspective and at an organizational level as well. So, I'm taking us back to that agent hub that I showed you a little bit earlier, and on the fourth option down that I kinda skipped over last time is a monitoring factor. So, from here, agent hub admins can go in and monitor the number of agent messaging, the feedback that people are giving in the activity that that you're seeing. So this is just a high level of overview of how, your line of business users are actually kind of thinking of how the agents are responding. So we can see positive, negative feedback right here. If you want to drill into actual conversation, like, why the agent did the way it or acted the way it acted. Right? From my city details, we've gone over sources and activities. But at the bottom of activities, we actually have this option down at the bottom to view trace explorer. So all of these questions and responses are gonna have traces in the background to actually go in and be able to kind of show you what's happening. So I can actually see, this exact trace, you know, what the message was, what got sent, what the query was, why the, assistant chose what it did, and the output that it's given. So your analysts are able to go in there and drill down and see, okay, why was the agent acting upon the way that that it was acting? On top of that, right in the platform, we can provide things such as evaluations. So, as you get these conversations and answering and responses, all of that can be tracked within datasets and you can build out evaluations on top of that with no coding or coding within the platform that you'd like. So as you can see here, we see an LLM evaluation. And here's a really good way to, build out how things are performing over time. So you can keep track of these and have these automated on the back end so that your agents or so that your analysts can trust that the agents are performing well over time at a at a monitored level. From an organizational level, what do what are some of the things that Dataiku provides? So we're gonna provide a lot of guardrails, safety guardrails, quality guardrails to make sure that things like PII detection and toxicity detection are all within within these as you are opening up these LLMs to your organization. A big thing that, within the Dataiku platform is cost monitoring. So we can provide kind of estimates of how things are acting from a cost level. These LLMs aren't for free. This is something that organizations are taking advantage of within the platform to keep track of what they're seeing from a cost perspective. Then the last thing and probably the most important in my eyes is the governance side of things. So within the govern node within Dataiku, what you're seeing here is a way to make sure that not only we're governing these projects from an overarching level, but that these agents and generative AI products that your analysts are creating, actually have eyes upon those. So as you can see here in a second, you will see risk matrix that we see. We'll see project reviewers and, how people sign off on the project. But as you can see here, right in here, I can see risk matrix for all my business initiatives. So I can have business initiatives across my, organization. And as you can see here, we've been kind of talking about a finance project and a finance chatbot so far, so I have my finance business initiative. As you can see here, I can see a risk value matrix on both not only the business initiative but against projects too. We can see who owns this business initiative, the projects, and the type and scope of the initiative too. If I wanted to, I can go down and find my project, which is this financial analysis assistant. We can see where this project is in its life cycle and not only from a project where I can see my sign off, reviewers, and approvers within the project, I can continue drilling down into specific governed models. That agent that I've been working with, this financial analysis assistant, I can drill in here and I can see what version of this model I'm on, what govern project it is, timeline attachments or anything that we'd like to see to it. Most importantly, what I think is you can drill into that actual agent itself and you have your model sign off. Right? So anytime your analyst goes in there, and makes an update to this agent perhaps, you're gonna have different levels of your organization. You wanna be able to go in there, provide feedback, or, provide an approval, based off of what they're seeing. So your IT operations, your risk and compliance, your business users, all of them may be testing out different things within that, agent itself, how it's performing, you know, is it keeping track? Is there monitoring on the back end? All of that, you can request feedback from those line of business leaders, and then get a final approval for that to be signed off and then be able to be deployed in production, all kind of within the platform itself. So, like I said, as your analysts are going in there and making updates, here we can go ahead and we can see your organization is keeping governance on this, not only from a, you know, model agent perspective, but all the way up making sure that's tagged to a specific project and a business initiative to make sure that governance is in place. So I'll go ahead and stop sharing and pass it back over to Conor for the last part. Awesome. Thank you, Cody. So a couple final thoughts here, and then we'll jump into there's a bunch of really great questions in the q and a. So couple last thoughts here, and then we'll transition into answering some of as many of those as we can get to is this final thing of the role of the analyst and how will our jobs as data professionals, as analysts change? You know, if you think about how this I've talked about this bunch already, you know, manually rebuilding reports, explaining a metric on a request, justifying how you did something, or constantly ad hoc, you know, here, do this, do that, etcetera, of building data models, building spreadsheets, whatever, to answer our questions versus, ideally, this new world of being able to direct attention to the right aspects that's in the data that's being provided, being able to create the quality in the structures and to validate the outputs of what's in there, but not, you know, having to create it manually every time. A little bit of that sort of like in the loop versus over the loop type of idea. And so, you know, one example that we've, you know, talked about or or is, like, today, it really is, hey, I'm, you know, a leader. I have a question that usually already has some back and forth. And then an analyst goes off and accesses data, uses, you know, again, whatever tools or skills they have access to to pull that together, create some results, to then send me a report, to create a presentation, to be able to then give that answer back to that business leader. And rarely is this an actual sort of like linear process. Usually, there's a lot more back and forth than this. You know, each one of these arrows goes back and forth repeatedly. And it's, you know, very, very challenging to be able to do that versus having these decision based systems where something like an agent has access to multiple models, multiple data sources that the end user, that business user, is answering data from the system while the analysts are validating those inputs, validating those outputs, not necessarily on a one by one basis, but being able to say, how do I know that this is good, this is bad? Having that judgment sort of at the point of time. So this is where we see this world going. You know, there's a number of questions. And, Cody, if you're down with it, I'd say let's just, you know, roll into some of this q and a unless there's any last things that you wanted to show. So I actually think one of the two things I was gonna show might answer some of the questions that people have, so. it might be weird. Let me do that. I will throw it over to Cody one more time, and then we will dig into somebody's q and a. And yeah. And feel free to just answer any of the questions in there via the tool while you're there. It makes sense. Yeah. So some of the questions are, you know, how are we explaining how the agent can perform? How are or not perform. How are we explaining it? How to make decisions? How are we saying what it can and cannot do? Right? So that's the new life of the analyst. Right? The analyst role that, Conor is actually talking about. So instead of, you know, writing a SQL query to get an answer for a line of business user, creating a dashboard, the new analyst scroll is actually monitoring the responses of the agents and iterating the process of creating these. As you can see here, this is the actual flow on that back end of the agent. What we see here is some little code, also coding tools and actual prompting within it, as well as creating those knowledge banks. As you can remember earlier, we created an unstructured and structured knowledge bank. We're taking information from our PDF, PowerPoints, Excel files, data sources, whatever it may be, and creating knowledge banks. The tools that the agents have access to, as you can see here, these are the tools that we've created. Now you can create coded tools. We have coded tools out right here. Someone earlier asked if we can run local MCP servers or bring in MCP servers from others. Yes, we do have an MCP server tool here. Then we have simple visual tools. It's also SQL queries using knowledge banks. Here are how your analysts are going to iterate on making sure that the agent itself knows how to use this tool properly. From here you can say, Hey, what the description is for that knowledge bank, you can run tests and see tool descriptions. Once you then create these tools, then you have actual visual agents or coded out agents itself. From this financial analysis assistant, as you can see, as you iterate on versions, you'll see multiple versions here. We have version one. Here is where we're prompting and instructing that agent on how to act. You're a decision support analyst, your knowledge based source consists of, your tasks includes. So we are prompting that agent on how to act. So I think there was a question earlier in this world of multi source, if you want an agent to choose one source over another in certain acts, you can tell it to do that here. And that's where the, analyst's knowledge comes into play. So they're providing the agent what they would typically do so the agent knows how to act. Right? Here's the tools that we provided, and then they can actually go in and actually test out that chatting right here. So just like I was doing from a production wise standpoint in agent hub, this gives them a quick, easy place to go in and and ask questions of the agent to make sure it's performing how they want it to perform. So hopefully that answered a couple of the questions as far as how do we make sure agents are acting the way we want them to and what tools they can have access to. That is once again the new rule of the analyst rather than, oh, hey. Let me go in and do a SQL query. Let me create a extract. Let me create a dashboard. So, I'll stop sharing. Hopefully, like I said, that answered a few questions and I think we have a few more minutes here or a few more. Yeah. So I'll throw this up. There's a number there's a handful of questions around sort of specifics there. So here's a couple of, Yep. QR codes for things that you can sort of see, a simple structured one or, you know, if you're especially on some of these agentic topics like we're talking about. You know, there's a handful of questions in here. Multiple people are asking you if your project will be available for them to download, Cody. I don't I mean, I'll let you answer that one. I do not know if I'm allowed to share this one yet. I can check, for you guys, if we are happy to, but, I do not know if it's available to be shared yet. So The I will say so this the one that he is talking about here, we have a number of our business solutions or Dataiku solutions that have agents built into them already that are available for download either in the tool or via the website. The one that he was showing here today is, I'm pretty sure, like, up and coming ones. I don't think it's available yet. But we'll put a we'll try to track down when that's released and send that out when we send out the the email, with the slides and the recording. A couple pretty quick questions, you know, like, a little bit more technical and easier to do, around things like MCP servers, external data, rest APIs, all of that type of stuff. All of those things are available as tools for an agent, provided you have the right sort of networking and and capabilities like that. And one of the cool things is as Cody was showing there with the tools for the agents is there's a lot of out of the box tools of access to a predictive model that's been deployed via Dataiku or MLflow or access to this data source or access to another agent and sort of doing that like agentic orchestration of multiple agents. Those are both available as visual tools within Dataiku, and we see that becoming much more and more prevalent Dataiku and everywhere of the ability to do these. But also there's always that ability to I can code or I can work with an engineering team to code access to or to create an MCP server or access an external one, an external API MCP, etcetera, etcetera. So there's a lot of flexibility in these, and this is one of the challenges of that AgenTek sort of orchestration, I guess, you know, topic that is growing as the ability to interact with all these different agents and data sources accelerates knowing that's what he was talking about there at the end. You know, which tool did it access? Did this agent pull up the right tool that I think it should access? How do I do that? You know, he showed it in the stack trace that you can look at. I did a query and here was what it did to respond to my specific query. But if I'm on the back of the house side, I wanna be able to look at all 1,500 times that that agent responded to a question today and what tools did it use, how did it do it, If it was asked the same question or roughly the same question, you know, 10 different times, did it give the same answer, or it was asked about a specific type of data, did it go to the same data source every single time, or are there's there are conflicts, and that's how we start to think about, you know, so there's in sort of inference and then over time, how are we structuring these? There's a lot that we will continue to sort of be able to work through, but having the ability to see all of that, you know, is a challenge. Another question, that I definitely wanted to say was I saw Tom's question around especially for smaller organizations, what does it take to accomplish this? And, it's the same, I guess, you know, not to be, like, cheeky. It's the same question for a big company and sometimes, honestly, harder for large companies as it is for small companies. The thing that I sort of the, like, sort of three key things that I see, you know, in being able to move forward for this is, one, is connecting and creating access to as much of your data source through, you know, simplified front end or, you know, ideally sort of like one access place for all of that. It's a combination of the sort of data mesh, data fabric type of ideas that we've had there, but also bringing together different user types into the same environment. That's I think the first one because as long as things are scattered across different teams, different tools, different everything, it's very hard to, you know, really start to unify and create this. The second piece there is empowering everybody to be able to work with data in different ways. It's great when you're at a company and you can hire a whole bunch of data scientists or have, like, a whole huge department of people working on this. And while it's great, it also has its own challenges. But, you know, when you're a small organization, hiring a data scientist, you know, maybe out of reach, let alone having a whole team of them. And, obviously, a solo data scientist is a really hard role to both fill and to perform to be able to sort of work in that environment. And so enabling everybody to have access to data and access to tools that allow them to work with that data, I think is the second really important thing there. And the third one, if you're hopefully doing this in an integrated environment for all those different user types is the governance on top of it to be able to look at who accessed what data, whether it's via an agent or via a different method. You know, how did the agent response to those things like that? Like, it really is those three types of things to really sort of, like, you know, have one place where people are accessing it, have everybody, regardless of skill sets, accessing it through that sort of simplified environment, And then having a governance on top of it to be able to quickly see who's using what, how, when, and where, both human and machine users of data, I think is how you can do that even at a smaller organization, because it's not it's not hiring consultants to do it for you in most cases. It's hiring consultants to help you set up some of those accesses. Right? Sometimes we we still need those external, but you can't outsource the data knowledge or that sort of working in there. Other Cody, any other questions? I know you've answered a bunch, like, in the chat. Are there any other that you wanted to hit? there's one just about, like, a specific integration with that, Zillow's Zillow's MCP server. So along with, like, any MCP server, or kind of really anything within the platform, integration, if there's not something directly out of the box in the no code format, you can probably code it within the platform. So I'm at I'm who would make a solid safe bet, that the MCP tool that I kind of showed earlier could be configured to, integrate with that, specific MCP server via code on the back end too. One of the the last comments, you know, I I like, I appreciate your challenge, Daryl, sort of saying, like, you know, not convinced that this adds any value for a seasoned data analyst. I definitely hear where you're coming from, and I think that, you know, this isn't a change of days or weeks or months. Right? This is definitely, like, a change of of, a long time, to be very honest. You know, we've sort of had in many ways working with data has been, largely unchanged for decades at this point in time. Right? Like, how we interact with the shore, the data has gotten bigger and the processing has gotten faster and, you know, some aspects of it have changed. But you think we're on the precipice of something changing over the next few years. And hopefully, it is not that you're, you know, you're sorry about that, David. We'll see. And we'll make sure that we do it, in the email, we'll send that link that maybe that's pointing to the wrong direction. Sorry. But, yeah, I do I I hear you, Daryl. I I don't think that this is an overnight change, and, you know, the idea of babysitting a bunch of agents isn't necessarily appealing to me. And and, hopefully, we're able to move through this to a system where, you know, the, like, low value, repetitive tasks of a data analyst are handled by agents freeing us up to spend more of our time on the challenging ones, on creating them, everything like that. I also think it's a lot more likely that we'll see people whose roles sort of revolve around being agent wranglers. I don't know if I have a better word for that. Rather than be you know, all of us becoming prompt engineers, I do think that there will be a subset of, you know, sort of people in this this data community that will work on those. But there you go. Alright. Well, we're a couple of minutes over, so respecting everybody's time. Thank you so much for all the engagement, and the q and a. Cody and I are both available. You can find us on LinkedIn. If we didn't get to your question and you wanna hit us up, you know, we are definitely happy to sort of dive in onto these topics, you know, in other formats and everything. But thank you very much, to Cody. I appreciate, you taking the time to walk through some of this and thank you to everybody that joined us here today. We really appreciate you coming, joining us, having this conversation with us. We hope you heard something new today, whether or not you agreed with it. And, you know, like I said, we always welcome that feedback. So hope everybody has a great evening, afternoon, morning, whatever time it is. Thanks again for.