Shawn Cordner (00:13.102)

Hey everyone, thanks for joining the Amplitude of Tech podcast. I’m Shawn Cordner, Chief Marketing Officer of Amplix. That is Monkey, my dog. You may have seen her in some of the podcasts past, so I just wanted to introduce her to everybody. Today we had David Vidoni on the podcast. He is CIO of Pegasystems. What’s interesting about David is that he is a CIO practitioner, but he’s also overseeing a company that implement AI into their products. So we get to hear both sides of the AI conversation, how to handle it as a practitioner, but also how to be a responsible vendor of AI as well. So this is a good one. Hope you enjoy it.

 

Shawn Cordner (00:59.064)

All right, David Vittoni, thank you for joining the podcast. How are you today?

 

David Vidoni (01:02.572)

Good. How are you doing today, Shawn?

 

Shawn Cordner (01:04.427)

doing great. Could just do a quick little introduction, tell everyone who you are and where you work.

 

David Vidoni (01:09.87)

Sure. So David Vedone, I’m CIO at Pega Systems. Pega is an enterprise software company based out of Wolfham, Massachusetts. We focus on workflow, decisioning, and AI.

 

Shawn Cordner (01:22.198)

Okay. So we don’t usually like to do commercials on this podcast. try to steer people away from talking too much about what their company does, but I feel like it’s going to be relevant to the rest of the conversation. So if you could maybe give us the quick elevator pitch on what that actually means, what you guys actually do. it’s okay if it’s a little bit of a taller building and takes you longer than 30 seconds, but let’s, just kind of get into what Pegasus actually does.

 

David Vidoni (01:46.646)

So Pegasystems is a 43 year old enterprise software organization, rooted in workflow, specializing in the… Pegasystems is a 43 year old company focused on workflow and automation. We focus on the enterprise. We have…

 

done extensive work in financial services, healthcare, insurance, you know, all about eight verticals. What we do is we address challenges related to organizations that need to get work done, work done efficiently, combination of people, of automation and integrating systems and bringing about the agility that the organizations need to survive and thrive in today’s world.

 

Shawn Cordner (02:33.164)

Okay. So I feel like your marketing department is going to give you an A plus for delivering the scripted pitch there, but let’s kind of dig into what workflow automation means. Can you give me like some real life examples? And since you’re a 43 year old software company, let’s start with before AI was a thing. And now that AI is a thing.

 

David Vidoni (02:51.118)

Sure. So workflow, very simply, I’ll put it in terms of case management. You and I have called companies, we’ve had issues with our accounts. We need to have things looked at. Maybe we need refunds or just putting charges on a credit card. When you call in, they’ll create a case for you that will kick off an investigation. It might be something that they can resolve right away. It might involve routing it to other people within the organization. At some point,

 

They come to a conclusion, either you have a good outcome on that and they refund you charges, or it’s something that they cannot deny. But either way, it’s work that needs to be tracked. It needs to be tracked throughout the organization, who has it, how long it’s sitting with them. There are many situations like this within companies where they need to make sure that they’re efficiently getting their work done and knowing where bottlenecks are and having opportunities to optimize them. So PEGA addresses that space by

 

providing a very robust set of tools to build systems and workflows that are tailored to the way businesses function, specializing for what they do well, what differentiates them with their competition, and not boxing them into a solution that is just off the shelf, you know, where they need to necessarily adapt. So that’s kind of the foundation of what we do.

 

where AI comes in and how things are evolving, we’re moving towards more of an automated world. We’ve been working for many years around automation, traditional automation, robotic process automation for systems that cannot be integrated with, some other automation, bringing together systems or just doing straight through processing and decisioning. AI takes that to another level because for steps that did require some analysis, some

 

decisioning and review that is not structured, is not necessarily repeatable. AI is nicely fit in there to help to bridge that gap and to automate those steps. And so we’re weaving those capabilities within our products. So it allows organizations, you know, those ones that they continue, they need to continue to run their systems and their businesses tomorrow and next year and the year after. They don’t want to throw out all of those.

 

David Vidoni (05:15.49)

those processes, they need them to run the organizations. They need the consistency and auditability and how they do it. They don’t want to have it being run where you get one outcome, one time you run it and you get another one another time and you have no idea why. So introducing AI, it points in the process that delivers repeatability, additional automation. gives them a leg up and additional efficiencies so they can continue to innovate.

 

and focus in the people in on the areas that are truly adding value and getting them away from the places that the tasks are somewhat repetitive and tedious.

 

Shawn Cordner (05:57.378)

Got it. And I feel like this is where we’re going to start getting into the buzzwords, right? So you’re using agentic AI in your software platform, is that right?

 

David Vidoni (06:05.39)

We are using agentic AI, we are using AI assisted activities. So guiding individuals through discrete work tasks might be the next appropriate step to take. It might be suggesting knowledge articles that they could give to their customers or, you know, helping them to do research so they’re informed to make the right decision. So it really spans both, both agentic as well as AI assisted.

 

Shawn Cordner (06:32.846)

One of the reasons that I was excited to have you on the podcast here is because you’re a CIO that I think is experiencing AI from both different directions, right? So you’re a consumer of AI and you’re someone who’s implementing AI into your business and into your platform, but you’re also running a business that goes to market with an AI product or at least AI integrated into the product. So I’m interested in exploring both angles of this. Before we get too far into that though, I wanted to just kind of understand.

 

When you’re sitting down and you’re talking to a client or a prospective client and you’re kind of looking at the landscape of their workflows in their business today, how are you helping them make decisions about where to start, what to approach next in terms of automation? Like how do you find the low hanging fruit or the most accretive initiatives that are going to have the most benefit to the business?

 

David Vidoni (07:24.054)

starts with identifying, you know, where are the bottlenecks, the manually intensive steps, the time consuming aspects of doing work. Maybe there are areas that, they can’t respond fast enough, or people just prefer to interact with automated bot or to do self-service. They want that instant response. Something like that would lend itself very well to an agent to assist them and guide them. And it’s not.

 

a consistent process every time. Everyone’s going to come in with a different challenge, different context. So to have the flexibility to adapt to that and personalize those experiences is very helpful. And also, you know, it increases the level of service that they can provide to their.

 

Shawn Cordner (08:13.902)

That makes sense. We spent a lot of time on this podcast and really all the content that we produce here at Amplix talking about AI in the contact center and how AI agents can deflect calls and help people get to answers faster. That is fertile ground for us to continue discussing, but I’d like to just get into some areas that we don’t more commonly talk about.

 

Aside from the contact center, what are some other areas of the business that those and technology leaders should be looking at for these opportunities for automation? Like for me, like accounting probably comes to mind, like AP and AR processing. I know marketing has a ton of use cases. I could talk all day about what we would love to do here at Amplix. Right. So where would you look in the business aside from the contact center?

 

David Vidoni (09:00.814)

So I’ll spend the different functions of the organization. We’ll start with finance since you brought it up. Accounting, accounts payable, accounts receivable. Even the process of someone requesting to something to be purchased, navigating some of these legacy systems. They start with some information that they need to collect before they even ask you what you want, like cost center. They have no idea. They have no idea what these things are. They just want a piece of software or they want to buy.

 

service, it’s an excellent place to put AI in front of it to guide them through that process. So, you know, they can say what they want and then the system can help fill that information in the background. And then that can be submitted to, you know, the other steps in the process to get the right outcome. Financial teams that are, know, public companies that are doing quarterly close activities.

 

There are very repetitive tasks that do require assessing, you know, that transaction in the context of what’s going on. Maybe there’s other factors that need to be looked at. It’s something that you couldn’t just solve by, you know, traditional automation. You need to make some judgment calls. Maybe you need to look at some thresholds and all of that. Those are excellent opportunities for a discrete task to automate with an agent.

 

and bring about better productivity and reallocate those folks to do something else. In the HR space, there are painful processes that happen throughout the year, annual reviews, Every year, I’m asked, what did I do this year? And you have to spend some time pulling that together. If you have an agent or AI assisted coach that could go and look at everything that you’ve created,

 

Everything that you sent out for release notes, announcements, emails, or even go against the systems that you were using to track delivery, they could cut that process down significantly and get you a good working draft to submit. Moving over into the legal area, there’s contract redlining. There’s just, you know, understanding complex contracts, being able to query them and ask specific questions and understand it on the…

 

David Vidoni (11:26.914)

You know, also in the finance area, in terms of, you know, analyzing data, understanding trends beyond what you would do normally in, in, you know, Power BI or traditional reports. You can, you know, ask questions of the data without having to build a report for every way you want to look at it. So, you know, those are, those are a few of the options and opportunities that.

 

that AI and agents can lend itself to. know, in terms of running things in the background, know, there are many things that you would like to know, but you wouldn’t know unless you were, you know, to look at report or check in on it. You could schedule those things to check in on those things based on thresholds and alert you when it’s relevant. So there’s just, there’s so many things across the organization from a business standpoint that could be.

 

improve significantly and they’re allowing you to do things consistently. It’s not relying on someone to remember to do it or if they have time to do it. There are things that could be scheduled and run on a set interval if certain conditions apply and then bring that to your attention so you could take action on them.

 

Shawn Cordner (12:36.28)

Yeah. You mentioned performance reviews. I’m about a month behind on completing my performance reviews right now. So Melanie from HR, if you’re listening to this podcast, I apologize. I’ll get to it eventually. As soon as David sets me up with some automated workflows to get it done for me. But I want to focus on what you just said there about scheduling the AI to look into, you know, specific areas of the business and then kind of create a report or dashboard.

 

and notify you in regular intervals. I love that idea of it’s a window into the business, right? Being able to not just see the data, but probably engage with the data in ways that you normally couldn’t engage with a spreadsheet, right? Or at least it would be a lot harder to do that. And I feel like that’s a use case that I don’t hear that much of, maybe because it’s more complex in the sense that you need to have your data ready to be able to do that. And you have to…

 

probably do a lot of integrations to make sure that those, the data flows from place to place, and then you have to be able to interpret that data. So just kind of talk me through like, where is that use case in kind of the order of priorities for most businesses and should it be higher and how can they get there?

 

David Vidoni (13:49.588)

It should be higher if there’s you know, costs associated with losing a customer. There’s a lot of energy in terms of, you know, getting a customer, keeping them. If there are things that you can look at and those things could be, you know, dots across the organization, maybe, you know, maybe they’re not using as much, you know, software, example, not using as much software as they did in the past.

 

They’re not engaging as much in some of the events that we’re putting on. The projects and other activity is going down. may not be a great indicator. Looking at any of those data points in isolation may not give you the complete picture. Building a complex report that joins all of that together is very tedious. It’s time consuming.

 

prone to breakage because if systems change and data changes, then that report will constantly have challenges to stay current. Using AI, you know, and being able to ask information, questions across that information, you’re not relying on, you know, rigid connections and joins and things like that to get your insights.

 

quickly navigate across those different data sets, those applications and have the agent help to make an assessment on the likelihood that that customer will renew or if they’re a flight risk. Those are types of things that are extremely valuable to put in play and it’s kind of…

 

All the benefits of integrating with systems without all the rigidness of APIs and joins. can interpret things just like you and I can interpret things provided it’s given the right instructions.

 

Shawn Cordner (15:48.14)

Yeah. Data is a funny thing, right? Because on one hand you think data is data and truth is truth and it is what it is, but there’s data can also be manipulated or it can be subjective in certain ways. You and I could look at the same report or the same set of data and we could draw different conclusions or interpretations of what that means. So I imagine that if you’re doing something like this, you need to run some sort of a regression analysis to see historically what those things meant and what the outcomes.

 

would have been and that figures into it. I also wonder, know, is there some danger of bias showing up in using AI in that way in terms of, the person coding it or the person training it and the way that they would look at data and the way they may interpret signals from data. Can that come through in the AI model itself?

 

David Vidoni (16:38.574)

could. We can also run into issues with hallucinations. What I find works very well is when you’re discrete things. If you’re asking it to do 20 things, I think the quality of the responses that you will get will break down. Just like if I was to ask you to go and do 20 things right now altogether, you might forget a few. I know I would. AI is in

 

some respects, not different. So if you can do one at a time and pull them back and, and then assemble those, you have a better chance of getting, you know, a result that’s of high quality and is consistent. I think the other thing that is really important as agents are rolled out and AI is used more frequently is the ability to test the quality of those results. And testing has to be done with different approaches and different tools. Old school testing, you know,

 

You had standard inputs, you got standard outputs, you would build an automated test for that, would check, everything was great. With AI, you put in a request with some inputs, first time you get this type of results, second time might be slightly different. You have to be able to flex with those variations, but still understand that it’s an accurate result and do it with a level of confidence, a level of confidence scoring. So I think any critical…

 

agent or AI process that you’re inserting in an overall process should have a level of quality checking because your inputs might change, the models you’re using will definitely change over time. You want to make sure things don’t like slowly drift and you’re not aware of issues with the results coming back.

 

Shawn Cordner (18:25.454)

That’s interesting. So how do you control or at least watch for the drift? mean, you must have some sort of, you must have some sort of benchmark that you’d have to be able to go back and reference, right?

 

David Vidoni (18:36.642)

You have to apply different, like I said, different types of tools that, testing tools that, if you have a known test dataset that covers a variety of different types of cases, you’d have to feed that through the process and, you know, using a scoring based approach, assess how close that is to what you expected. And that will give you a level of confidence that.

 

whatever you’re running is maintaining the same level of quality in the response.

 

Shawn Cordner (19:07.564)

You’ve used the word confidence twice now in this conversation and earlier in the conversation, you said something to the effect of AI using judgment or use, you know, judgment was the word that you use in the context of AI. I’m wondering, you know, can AI truly have judgment? maybe the easier question to answer is, when is it appropriate to have AI, agentic AI running in a fully autonomous?

 

kind of way versus having a human in a loop.

 

David Vidoni (19:38.944)

I think when there are situations where it’s running reliably over a period of time, you you’ve assessed the results. You don’t see any surprises. It’s consistent, you know, and I would say, you know, again, it’s handling, you know, small, discrete things and you can evaluate each of those decision points. If you’re not seeing any anomalies or drift over time and you have validation processes in place to check that.

 

It is probably okay to let that go and then just be alerted of exceptions that get highlighted. However, you know, there are many things that you still want to have people in the loop to, you know, evaluate. I would imagine no one would be pleased if they were, you know, applying to a position at a company and all decisions were made by AI. That would

 

That things could happen on so many levels, you know, we would never want that to happen. You don’t want to have someone be evaluating those recommendations before any decision was made and they have an opportunity to look at, you know, the applications and other things. So there’s so many situations like that in a business process that you want to have humans in the loop to evaluate. You know, the decisions that AI is making is only based on

 

the information that’s available. So if you have questionable data quality, you don’t have all of the information that is accessible. You absolutely would want to have someone in the loop to make sure that, you know, something wasn’t missed. Maybe there’s, you know, context outside of that process that needed to be factored in before our final decision was made.

 

Shawn Cordner (21:24.046)

Yeah. It’s interesting. You mentioned hiring as one of those processes, right? That it might not be a good thing. So Amazon, I’m pretty sure ran into a problem where they, put AI in front of the hiring process. So it would scan the resumes that were coming in and it would make decisions based off of that resume and the position that they were applying for. And you had to get through that filter in order to get to a human being and actually get into the process.

 

which I think happens all the time. And what they ended up finding was that the AI was trained on historical data of who was in those positions. And of course, historically who was in those positions were white men in those management positions. And so it started blocking all females from getting through into the hiring process. And so they caught onto it pretty quickly and they obviously made that adjustment, but that’s an example of why you wouldn’t want AI in that process.

 

Conversely, there’s a famous study in social science that they did. The guys, the Freakonomics guys talk about this. I don’t know if you’re familiar with their work, but they did a study on, I think it was people that were applying or trying out to be in an orchestra of some kind. And so they did it where they could see the people and then where it was blind where they couldn’t see the people. They ended up realizing that people were making bad decisions because their human cognitive biases were getting in the way.

 

when they could see the people and talk to the people. And when they just purely heard the music, they made better decisions in who they were going to bring into the orchestra. So that’s an argument for maybe why AI could be better in the hiring process. And there’s other studies as well that say people just don’t do that well hiring. It’s almost like throwing a dart would be better because you get a false sense of confidence. Like, I’m good at reading people and you know, I’ve

 

you’re drawing on experience and things like that, Where, where AI can be a little bit more objective in the data, I think.

 

David Vidoni (23:27.02)

Yeah, but I guess my philosophy is, know, anytime you take your eye away from anything, you know, problems can happen. So to the extent that you can use, you know, capabilities like this to enhance the data you have, some insights, I think you will get to better decisions. So it doesn’t have to be an either or thing.

 

Shawn Cordner (23:45.966)

Sure. Multiple inputs are always better, right? So getting back to trust for a second, you talked about having the model running for a period of time and analyzing it and making sure that the results are predictable. What else goes into trust? mean, how do you even get to that point? You must have some implications around governance and around observability and explainability, right? So what are the other components of trust in AI?

 

David Vidoni (24:13.742)

It’s just that, you know, an audit trail of what decisions were made and why, you know, being able to monitor to make sure that all of those critical components are operating at all times. And, you know, you didn’t have, you know, three or four things that were running and then maybe the second or third one didn’t run for some reason. Maybe the model was not available or there was a glitch that could easily just continue to fall through and get processed in a way that you don’t expect. So, you know, having.

 

Having that transparency, not having to guess what happened when that was processed is really important, particularly when you’re dealing with regulated industries, industries that are subject to SOX compliance and regular auditing. You can’t just have something processed and have it be arbitrary and this time it comes out this way, this time it comes out the other way. And I said at the beginning, like,

 

Companies want to provide consistent service. So they need to ensure that they have, you know, consistent set of business policies being adhered to and executed. you know, by having that transparency, you know, seeing that in the audit, understanding exactly what ran and why, I think gives organizations the confidence they would need to know that it’s working right.

 

Shawn Cordner (25:34.382)

Yeah, I want to dive into the implications of regulated industries, but before I do, just want to put a bow on this part of the conversation. So is there anything else that you would say makes agentic AI enterprise ready or maybe the other way around making sure that an enterprise is ready for agentic AI?

 

David Vidoni (25:55.178)

On the, on the second part, I think that there’s a lot of work that has to be done, change management to make sure an enterprise is ready for AI. You know, introducing agents into a process, having them make some of the decisions, doing some of the analysis. It requires a level of comfort and trust and being comfortable to step away from that particular task. And, you know, that might be unsettling for some who might see that as a

 

first step of having their roles fully automated away. But, you know, on the flip side, there are so many things that people do today, day in, day out on their roles that are repetitive, they’re tedious, they don’t enjoy doing. And because they have to do it, they don’t have time to do other things that they should be doing. So I do think it is opening up an opportunity to, you know,

 

allow them to take on more interesting and value-added activities. That needs to be communicated and make sure that’s understood within the organization. And then also, you know, just understanding how to work with an agent as opposed to another person. You know, if there’s an issue, what do you do? Do you go to the agent’s manager? Do you, how do you interact?

 

with an agent if you have a question, if there’s an anomaly, is there support for that? You you need to think about things like that, because it’s not always happy path processing. Strange things happen all the time and, you know, you need to be able to anything that might come up and have a way to do that consistently.

 

Shawn Cordner (27:32.13)

Yeah, you said change management and we had Gary Sorrentino, who’s the CIO of Zoom, global CIO of Zoom on the podcast a few weeks ago. And he was talking about human change management as opposed to technology change management. Right. And I think that you’re saying both, right? You need to really think through both. And the human part of it is so important. In my career, I’ve been involved with a number of large software projects and rollouts and as a business stakeholder. And the ones that fail, like I worked for a company that

 

invested like seven figures, mid seven figures in, in like early 2000s money in an Oracle system. And they ended up having to rip it out like a year later because people just didn’t use it. didn’t, the salespeople wouldn’t use the CRM. The finance people never got comfortable with that module, the accounting module. So, and that was because it was just, it was poorly communicated. No one really understood the why there there’s a guy in the marketing space, Simon Sinek.

 

And he always says, what’s your why? And the importance of understanding where a business is going and the why behind the decisions that it’s making is I think underrated and a lot of business leaders really don’t think about, you know, it’s like, hey, we’re making this change and you’re going to do it because you work for us, right? But people need to buy in because you’re never going to get adoption if they don’t buy in and get aligned on what the goals are.

 

David Vidoni (28:55.426)

Yeah. And that goes beyond just AI. Any technology change, any process change, if you do not have buy-in, you run the risk of having it be either a failure or not getting close to the ROI that you hope for.

 

Shawn Cordner (29:09.196)

Yeah, absolutely. So let’s dig into regulated industries. What are some of the unique challenges of implementing AI, specifically agentic AI into workflows in heavily regulated industry?

 

David Vidoni (29:22.836)

You will have some situations where it’s just flat out prohibited. So, you know, it makes it very difficult. So that’s kind of a full stop. For other organizations, you need to have full explainability on what was done, what data was used in the decisioning, what the decisions were, under what circumstances. know, so to be able to go back in time for any and all, you know, decisions that were made related to work that was being processed is very important.

 

Making sure that I mentioned this before, know, as models evolve and change that you don’t have drift, you don’t have differing results and they’re happening and you have no idea that those are causing a negative impact to the quality of the work. So what controls do you have on ensuring that, you know, those are consistent and within the level of acceptable quality and consistency?

 

Those are some of the things, know, some of the other things that need to be factored in. How are changes introduced? How are things managed that, you know, all go into, you know, this process, you know, that needs to be factored in so, you know, you don’t have other things that might be impacting the output and the quality of that. All of those changes need to be tracked with the appropriate…

 

evidence that they’ve been tested and there’s no opportunities for unexpected or unpleasant changes to be introduced. It could be security threats or unintentional errors that made their way in.

 

Shawn Cordner (30:51.84)

Yeah, I’m wondering if, you know, having AI in a healthcare environment, for example, increases the potential for data leakage, which makes you at a higher risk for violating HIPAA regulations, for example. And if that’s the case, I also wonder, are there implications on the cyber insurance side for that?

 

David Vidoni (31:13.24)

There are use cases that are incredible for healthcare. Being able to assist someone in radiology to look at images. Doctors to be able to summarize notes they had with the patient so they could send them out. And every situation, those are things that can be reviewed by a human to make sure that it’s accurate and expected. In terms of security, I sincerely hope I would not expect that they would be putting this into

 

How about chat GBT? you know, making sure that the models that they have are the chats and everything is fully secured and is not available for sharing with, you know, safeguards for making sure that, you know, no one outside can access it without permission and all of that. So, you know, operating at a zero trust model. There are also some emerging standards that are coming out to certify AI.

 

models that also come along with additional insurance safeguards that could be layered on top of it, similar to what is already available today for cybersecurity insurance, but more specific to AI.

 

Shawn Cordner (32:27.138)

The federal government has largely been absent in terms of setting any kind of regulation around AI, there are frameworks, So NIST being one of them, or CMMC. I’m wondering, are the regulatory bodies themselves up to speed on AI and are they updating their requirements and regulations as well?

 

David Vidoni (32:47.544)

I think everyone is just trying to stay up on all of the changes. Are there times when, you know, things lag behind? I’m sure. I don’t know how they wouldn’t. I mean, things are developing by the day and during the day. So I’m sure there’s a lag there, but I think what probably will make sure that we have these standards will be companies. Companies want to make sure that they have trust and, you know, customers trust them and…

 

that they’re doing things with integrity and explainability. you know, having these types of certifications, being able to, you know, point to, you know, ISO, you know, 40s 2001 or other standards shows that they put energy into this. It’s a trustworthy process. It’s a very rigorous certification, you know, or, you know, adhering to NIST or other frameworks.

 

be a differentiator for them. Not having that will slow down their ability to sell and be competitive in the market.

 

Shawn Cordner (33:54.27)

So let’s dive into you as an AI supplier or having AI in your product for just a second. So are there any unique challenges to introducing AI into a product or going to market with an AI based product that you’ve had to consider that are above and beyond the type of strategy and governance and security measures that you would have put in place as a user of AI?

 

David Vidoni (34:20.782)

For us, the basics of wanting to have it certified like our other products, making sure that it’s integrated into the things that we already have. We’re not building parallel security models. We’re not having to take copies of data and put it outside of the system because the architecture can’t handle that. We need it to be native and integrated into the solutions because if it’s not and it’s on the outside,

 

Every time you want to change something, you have a few additional hoops that you need to jump through. And I’m sure there are several other vendors out there. I’m not going to name names, but there are other ones out there that probably don’t have architectures that lend themselves very well to adopting AI easily without going through, know, magic tricks and, you know, maybe, you know, additional layers to, you know, give a level of, of agentic.

 

Over the long haul, think we’ll hamper innovation, hamper speed to market if you don’t have something that is natively designed to embrace and work with that.

 

Shawn Cordner (35:34.254)

Go ahead and name some names because I’m waiting for my viral moment here to make this podcast big. let’s, let’s start with the concept of it. No, I’m just kidding. So we’ve had a few AI experts on that have made it a point to talk about the risk of AI in the supply chain. So now that you’re part of the supply chain, how do you look at supply chain risk and do you do anything differently in terms of vetting your vendors and

 

understanding how they’re implementing AI in their systems.

 

David Vidoni (36:04.242)

We do, we make sure that they have the requisite certifications, know, understand what frameworks they’re working against, what evidence they have. It’s honestly, I don’t know if we fully kind of internalize the full impact of what things might be coming. You know, if you build agenda capabilities in a system, how do we know that there’s, you know, redundancy in that? If a model is not available for some reason.

 

Maybe there’s an outage, you know, how does the system behave? You know, has that been properly accounted for in the processing? What if it’s down completely? You know, can the system continue to work or are you dead in the water? You know, like, you know, one of the public cloud providers, you know, your application is all on the cloud and it’s down and you don’t have, you know, failover, you’re dead. So same might be said for agents, you know, what happens if those go down and you’ve

 

You you’ve downsized or maybe you didn’t upsize in the first place. You were fully reliant on these. How do you continue to operate? What happens if, if, know, some vulnerability is injected in? Do you have the right checks in place to make sure that code is being tested for vulnerabilities afterwards? Or are you assuming that, you know, you have a step in the agent to do that for you and everything’s good? There’s a lot of assumptions that people make when they build systems and we just need to, you know, really go through a deep.

 

analysis of what if scenarios to make sure that it’s actually behaving as expected. And, you know, no different than if you were to do a tabletop exercise or, you know, just, you know, deep testing with adverse conditions. I think the same should be done with, with AI that’s inserted and agents and people should understand how things are going to be when bad things happen.

 

Shawn Cordner (37:53.784)

You had mentioned that you get your products certified. What is that certification and should the people listening to this look for that certification in their supply chain?

 

David Vidoni (38:03.662)

So I had mentioned earlier ISO 400001, this to certify AI and there’s a whole series of controls associated with that. you know, it involves bringing in an outside auditor, doing a thorough review of it and, you know, understanding how all of those controls are met and making sure that it’s not just, you know, for the one time that you have it tested, but you know, there’s…

 

ongoing processes to ensure that that will be adhered to.

 

Shawn Cordner (38:34.112)

So we talked about workflows and if I could zoom out for a second and let’s just talk about modernization, right? Modernization of the tech stack, but also leveraging technology to modernize the way that you’re actually doing business, the way that the business functions. Clearly that’s an ongoing thing that people are grappling with, right? How do you escape tech debt? How do you, you know, adopt newer technologies without disrupting the business? But what I’m most interested in is how do you do this, like approach this modernization?

 

without creating more complexity in the business.

 

David Vidoni (39:07.38)

many, many tools out there. We offer some, you know, there are ones too for different, you know, technologies where, you know, can analyze a code base. Maybe it’s a COBOL, know, mainframe COBOL system in an insurance company or, you know, somewhere else. Been around for, you know, 30, 40 or more years. You know, it’s kind of like, like asbestos, like nobody wants to mess with it because it’ll just, you know, create all sorts of problems. However, you know,

 

It’s a limiting technology. The availability of coders to continue that is scarce. So it just limits your ability to innovate. There are tools out there that can analyze that, bring that into a different model that describes what is being done by the application. What fields are being processed, business rules, all of that.

 

Those can be imported into new technologies, next generation technologies. We offer some, but there are other options too. And so that, it provides a company opportunity to not spend months and months and months doing deep analysis on what was built and why it was built. And, you know, there’s all sorts of tech that in there too. You can do that with, with AI tools, understand it, describe it. Have an opportunity to, you know, even reimagine it. So maybe.

 

some of the steps that were there, either to be there because that was the best thing that you could do at that time. You know, now it might be something that could be handled by an agent. It could be something that could be, you know, is more conducive to straight through processing and integration. So there are opportunities to reimagine these things in ways that just simply were not possible. So I think today there are so many opportunities for organizations to move beyond those and to step into the future.

 

and to continue to innovate and evolve and make AI an agentic part of that.

 

Shawn Cordner (41:05.752)

you suggest the technology leaders listening to this go about getting budget for these kinds of initiatives? Like what’s the process? What are the magic words that they need to say to the finance department or the board to get behind that largest scale of an initiative in terms of what the cost, also the time and resources that will be dedicated to large scale change?

 

David Vidoni (41:28.674)

What I’m, what I’m seeing is more and more I’m hearing from folks, know, the appetite to do these massive projects and hope for the best and good things will come out on the other side without doing any initial pilots. know, the appetite for that is next to nothing. You know, it’s very low cost to do a POC, to take, take a module that, you know, is maybe complicated. there’s dead code, other things.

 

things that you you weren’t able to do anything with before, able to extract that out and transform it and put it into the modern era. And then, you know, being able to show that will give more confidence and also better insights into the level of effort that would be associated with taking that to the next level. The other thing too is you don’t have to take these things as big bangs. You you could peel off portions of a system and modernize those in phases.

 

And the nice thing about that is it brings down the risk. It’s easier to focus on. I’m sure you do learn things in subsequent phases so you can get better at it. And so there are ways that you can do that without totally taking the business offline for that conversion. And those things can be done much more rapidly. So there are different ways to attack it. What I would say though is probably the one that I would not

 

advocate for is that big bang where everything’s taken offline and the big migration, if there are alternatives to eating that elephant one bite at a time.

 

Shawn Cordner (43:05.71)

But on the other side of that though, which your approach sounds completely logical and intuitive, but the other side of that is the famous MIT study that everyone’s arguing about 95 % of AI pilots fail to produce an ROI. I think the argument is about, there’s a few ways to push back on that, right? Number one, the methodology of the study is often questioned. Number two is, is it really a failure or was it a learning process?

 

And number three was, was it ever intended to generate an ROI to begin with? Right? So I think those three things might be related, but what do you make of the pilot to scale gap? And regardless of the MIT study, how would you tell someone to go about ensuring that they’re going into a pilot that if they intended to scale has a good chance of scaling?

 

David Vidoni (43:54.862)

think any of these early pilots are nothing but experiments. You have to understand how it works, what the boundaries are. As mentioned before, how much you’re asking it to do in one pass. There’s definitely higher success doing it in smaller pieces and then chaining them together. Understanding what works well, what doesn’t, that’s going to be what you’re doing in your early pilots. You might have some successes here and there, but…

 

More often than not, some things are going to work, some are not, and you’re going to walk away with low confidence that this is something that can work consistently at scale. So you need to refine, you know, the right level of scope of AI for whatever you’re attacking. then through, you know, other means, doesn’t have to be all AI could be, you workflow or some type of standard process to be able to execute that in a succession.

 

to be able to process it. How that scales up is, know, when you have those controls, the quality controls to ensure consistency of the monitoring, employing different technologies, not everything has to be handled identically. There are so many valid ways of tackling problems. It could be a combination of AI and traditional automation. So you need to understand the boundaries of it. You know, it’s not, you know,

 

You have a hammer and everything’s a nail. You need to use the right tools for the right use.

 

Shawn Cordner (45:27.214)

You called it an experiment and I just want to dig into that word for a second. Do you think that they should treat it like a scientific experiment in the sense that they go into it with a hypothesis and they design the experiment in a certain way to eliminate variables and try to predict what the outcome is going to be and then kind of analyze the results and the variance between the expected outcome and the actual outcome in the

 

interests of quantifying the value of the pilot so that they can leverage that for future predictions and pilots, but also to go back to the board and show good stewardship of the money that they’re investing in the experimentation.

 

David Vidoni (46:09.806)

I feel like that process, I’ve never thought about it that way, but as you were describing it, just felt very over-engineered, complicated. Sometimes there’s this value in jumping in and seeing where it will go. I was talking to someone yesterday and they’re like, you know what, you should start with the end goal and what you want. And then, you know, from there, what would be, what is the first thing that would need to be coming together to get that result?

 

Shawn Cordner (46:19.022)

over engineers.

 

David Vidoni (46:39.2)

step back, step back from the end goal to understand all the fundamental pieces that are needed to bring about that solution. I found that kind of intriguing. Maybe I’m doing that unconsciously, but I thought that was an interesting way to perhaps tackle, you know, some problems like this. But I think, I think it needs to be less structured. think, you know, providing it, you know, guidelines, you know, what you understand the constraints to be, what, you know, it should be looking at is probably a faster way to innovate.

 

And the way things are going, change is going to just be going faster and faster. I don’t know if people can take the time to go through such a scientific method for each and everything that they’re trying to do.

 

Shawn Cordner (47:22.766)

Sure. was DaVinci that said simplicity is the ultimate sophistication, right? So sometimes just keeping it simple is the mark of doing things the right way. I want to just take a quick left turn. I want to talk about AI in the dev cycle and how vibe coding may be changing that. Are you guys using vibe coding or using AI in your development cycles?

 

David Vidoni (47:44.738)

We are using AI in our development cycles. are, we’ve introduced some, some vibe coding capabilities as well to, to our experiences building applications, imagining them, ideating on them. And we’ll have increasingly sets of functionality that, that does that. I think is absolutely necessary. It is, it is nice because it allows you to quickly accelerate on, on discrete.

 

pieces of a system at a given time, whether it’s a screen, whether it’s a process, what have you. And, you know, doing that in pieces, I think allows you to assess what was done for that thing. You know, and you just send a bunch of instructions and a whole bunch of things happen in the background. You don’t know what happened. You don’t know how they happened, whether it was right or wrong. may be apparent, maybe not. It leaves some…

 

misstering some doubt and then over time, if everything was done like that, you might have no idea what to do if there’s a problem. So, you know, things can get away from you. So I think there’s absolutely value of doing that in a controlled fashion and iterations on things. And that’s where I think, you know, we’ll get efficiency of scale. And as, as that becomes more of a norm and comfortable, people understand it and it’s repeatable.

 

You will be able to chain more and more things together, but I would start, you know, with the atomic things or a few things and then go out from there as opposed to just a simple instruction that does a whole bunch of things and you have no idea what happened.

 

Shawn Cordner (49:19.168)

Any other tactical advice for people that might be considering introducing that into their development cycles?

 

David Vidoni (49:25.106)

I would say, you know, obviously play around with it, experiment without the, you know, hypothesis. I would probably also suggest, you know, getting some outside guidance on, you know, the right and wrong ways to do that. Because it’s more than just, you know, jumping in and in vibe coding, you know, there’s this whole change in the development process and the release process. There are probably roles that, you know,

 

will emerge out of this type of development that don’t exist today or don’t exist at the organizations today. So I think, you know, having someone come in with an outside lens to give some insights into, you know, how all of those things to come together and how you produce software is super important because otherwise you will learn the hard way. It will take much longer.

 

and you probably won’t be happy with the R1.

 

Shawn Cordner (50:23.63)

I’ve seen some stats, I can’t recall them off the top of my head and I don’t have them up, but that developers themselves when asked said that AI is speeding up their development cycles by say 20, 30%, something like that. But then when actually tested, they found out that it was taking them 20 to 30 % more time to actually do the same amount of work. Some of that might just be learning curves, right? They’re still, it’s clunky, it’s not, they don’t have their 10,000 hours of vibe coding yet or whatever.

 

But you I wonder like, you seeing an appreciable impact in productivity or the amount of time that it takes to get a release out the door?

 

David Vidoni (51:00.686)

We are seeing situations where people are absolutely getting value. They’re doing amazing things so much faster, things that would have taken months, taken a few days to do. So absolutely, that is happening. I’ve seen it. I’ve seen the results of that. To your general comment, people are saying, I’m getting 20, 30%. It’s anecdotal. I asked, well, how? How are you measuring that? Well, I…

 

It’s my gut feel. I think, you know, very soon we’ll have to have more objective ways to, to measure that. And it can’t be lines of code generated. Cause lines of code is just that it’s, it’s more stuff, more stuff that can go wrong. has no direct correlation to functionality and actually, you know, over time we’ll make things worse. So.

 

It’s going to be really important that any stats are backed up by data, credible data, not your gut feel and factor in, you know, the actual features that are being developed, defects that are being created and overall time that it takes to bring everything together to get it out the door.

 

Shawn Cordner (52:17.134)

I don’t want to ask this about Pegasus systems specifically, but just your viewpoint on how this is going to impact the industry broadly. You mentioned that it’s going to create new positions, but obviously one of the things that people are talking about in the media is how this is going to be an apocalypse to coders, right? That no one’s going to have a job at the end of this and AI is going to do all the coding. So I wonder, what do you think is the reality of

 

of what this looks like in three years. It’s probably in AI terms, that’s a lifetime, right? It’s impossible to predict. But let’s just try. What do you think is going to happen? Are we going to see the same number of people doing incrementally or exponentially more work and maybe creating some new positions or shifting some positions? Is there going to be a net loss?

 

David Vidoni (53:03.278)

I don’t know. mean, if I could predict the future, I’d be, know, betting on the stock market on all the wins right now. But the way that I do see things going is, yeah, roles will definitely change. What you’re doing today is going to be, you know, very different than what you’re doing in six to 12 months. You know, you’ll be interacting with, with vibe coding experiences, with agents that are doing discrete activities.

 

Do I think productivity is going to go up? Yes, I do. Do I think that overall output will go up? Yes, I do. Like I said before, other roles will be needed. Probably need, you know, because you’re creating more things, there’s more pressure on downstream activities for validation, for quality. You still need to have someone look at the actual usability of the products that are being created. Those things are going to be happening a lot more rapid. So I…

 

I sincerely hope that the smart companies are going to be the ones that are focused on improving productivity and output. And they’re not doing this for simply a cost cutting measure. Cause I think if you’re doing it for that purpose, you’re probably focused on the wrong things. Yeah, that’s kind of my thoughts in general. I do think there’s absolute needs for many roles and new roles, like I said, will come out of this.

 

Just like when the internet came around, people’s roles changed and a lot of new ones were created and things continue to evolve and innovate. So I don’t really see that changing with us.

 

Shawn Cordner (54:41.678)

Let’s just wrap up with a big question for the technology leaders that are listening to this. Do you have any advice or tips or guidance of how they can go from their current state to a more modern, more efficient backend process workflows? How can they streamline their business? Any advice for them?

 

David Vidoni (55:06.164)

I would say, you know, starts by immersing yourself in these new technologies. They are going to be the way that things get done now and in the future. And you need to constantly be looking at all of the developments, the trends. So, you know, whether it’s, you know, joining a tech group and, you know, listening to speakers talk about it, you know, connecting with podcasts, you know, working with a consultant.

 

It’s not something that you do this weekend, you check off the box and you’re set. know, information becomes stale very quickly. Capabilities change constantly. Something that does not work now, you could come back in a month or two months and it’s completely different. Or there’s something brand new that emerges. you have to, your ability to learn and adapt is going to be, you know, under extreme pressure much faster than anything else that we’ve seen.

 

And you just need to embrace it and lean into it. And the reason is, you know, these, these are incredible capabilities that were unthinkable just a few years ago. And it’s only going to become better and you’re only going to be able to do more than you. It’s the worst right now it will ever be. It’s only going to get better. And, and, you know, there’s, there’s just so much opportunity for the things that you, you want to do, what you couldn’t do. These technologies are going to enable you.

 

to be able to do those things in a much better way and much faster. It’s really upon all leaders to understand this, embrace it, and make sure that the systems that they have today, the data and the staff are prepared for this change because otherwise it’s going to be quite painful to make that pivot.

 

Shawn Cordner (56:55.638)

And given the advice that you just gave, I believe that you’re president of the Boston chapter of SIM. Is that right? So do want to give a little plug for the local SIM chapters of the people that are listening?

 

David Vidoni (57:02.776)

That’s correct, yes.

 

David Vidoni (57:08.386)

Yes, so I am the president of SIMB Austin. I’ve been with the organization for 10 years now. It is an amazing group of tech professionals. We’re about 400 people strong and growing. Amazing events throughout the year. I think we hold about 60 events. So great opportunities to network, to learn, also to give back. So whether that’s participating in committees, whether that’s getting involved with outreach.

 

One of our pillars is actually doing outreach and funding some outreach partners who can help provide a robust tech talent pipeline through alternative means. So we do many, many great things. So it’s a great place for you to build your external network and grow. And if you want to learn a thing or two about AI, we have many sessions for that as well. So love to have you.

 

Just go to simboston.org and you can learn more. if you want to join, there’s a link to join for membership.

 

Shawn Cordner (58:11.914)

And you and your representative from your chapter are going to be at our tech leader conference, April 30th, April 29th and 30th on Cape Cod. And you’re also going to be speaking on one of our panels as well. Aside from that, where else can people find you?

 

David Vidoni (58:26.99)

You can find me on LinkedIn. I attend many of the SIEM events in and around the Boston area, as well as some other CIO focused groups. So, you know, I’d love to connect and learn more. And hopefully we’ll see you in the Cape in about a month.

 

Shawn Cordner (58:42.668)

Awesome. David Vudoni, thank you so much for your time and expertise. Appreciate it.