The Legend of I-SPY 2 - Part B

Judith: Welcome to Berry's In the
Interim podcast, where we explore the

cutting edge of innovative clinical
trial design for the pharmaceutical and

medical industries, and so much more.

Let's dive in.

Scott Berry: All right, welcome everybody.

Back to part B of I Spy two.

I hope you got the episode, the previous
episode where we are visiting here with

Don Berry and we are revisiting the I
SPY two trial and we left off, uh, where

he's using MRI markers, auxiliary markers
to speed up the adaptive algorithm to

model the rate of PCR six month PCR.

And he was now talking about
one particular application where

the time machine is so valuable.

So how, what is a time
machine in a clinical trial?

Don Berry: So let me tell you,
in the context of ice I two, um,

we're, uh, 2014,

Scott Berry: 14.

Don Berry: uh, we've got, uh,
several arms in the trial.

Um, one of the arms.

Uh, used, uh, Pertuzumab, which
was an anti her two therapy.

It was used in addition to Trastuzumab,
uh, also called Herceptin, um, in

her two, uh, positive, uh, women, uh,
that is with her two positive tumors.

And, um, we had other therapies as well.

The standard arm in her two positive
disease included Herceptin and Paclitaxel

and, uh, doxorubicin, um, in this
six month period that we talked about

it, six months of the, of therapy.

Um,

and it's, uh, it, it so happened
that, um, gen, uh, Genentech.

Uh, had been investigating Pertuzumab,
uh, in her two positive disease.

Um, and there was a, another trial,
another neoadjuvant trial that was

comparing it plus Herceptin, uh, to just
Herceptin plus the, the chemotherapies

of, uh, uh, paclitaxel and, uh,
doxorubicin and Cyclophosphamide.

Um, so,

uh, the, uh, FDA gave accelerated
approval to Genentech's Pertuzumab.

Scott Berry: to ext

Don Berry: And that meant that,
uh, Genentech could sell the drug.

Um, and they had obligations about
coming back with, you know, the

longer term endpoint, but it was based
on an an, an improvement PCR rate.

Um, and our physicians and our patients,
our patient advocates, uh, and our data

safety monitoring board said You have to
drop the standard arm for those patients.

You can't give the standard arm
because it doesn't include pertuzumab.

Um, and I said, okay, we'll do that.

Scott Berry: that.

Don Berry: Um, we did it

by zeroing out the randomization
to the standard arm.

Uh, we luckily had pertuzumab in the
trial already as one of the experimental

arms, and so we were able to, uh,
sort of compare to Pertuzumab, but

there were only a few patients that
were in, and so it wasn't a big deal.

So I went to Scott and I said,

Scott Berry: said,

Don Berry: what am I gonna do?

I, I'd like to be able to use the
controls that we already have in

that, in the her two positives that
did not get pertuzumab, that some

of which still don't have surgery.

Um, and I'd like to use them.

For control.

Um, but I don't want to keep using them,
you know, for, for five years from now

when, uh, other things will have happened
and the disease may have moved on.

And so maybe I should discount them
somehow for, uh, the, um, for the amount

of time that they've had in follow up.

And, you know, when they
entered the trial, what do I do?

And Scott said,

Scott Berry: said,

Don Berry: you know,
I know how to do this.

I did it.

I did it in baseball and in golf
and in hockey, uh, and Scott,

uh, to you, the time machine.

Scott Berry: Yeah, so we in in sports,
there's the age old question of how do

you compare players of different eras?

How do you compare a Mark
McGwire to a Babe Ruth?

How do you compare Tiger Woods to Jack
Nicklaus They never played together.

The beautiful thing about sport is
they didn't play together, but they

played with players that played
with players that played with

players that did play with them.

So, babe Ruth May have played with Jimmie
Foxx They overlapped in their careers.

Jimmie Foxx overlapped with Ted Williams.

Ted Williams overlaps with
Mickey Mantle and Reggie Jackson.

And Hank Aaron overlapped a lot of people.

There's a bridging.

Over time of players at different eras.

And if you can estimate the effect of
time on the players, sports is almost

a little bit more complicated 'cause
players age themselves, drugs don't age,

but that if you could estimate the effect
of time, you can isolate the individual

effects of different players and compare
them even if they never played together.

Well here in a brand new kind of trial,
a platform trial, this exact problem

comes out where arms are in the trial
at different times and now you want to

compare an arm to patients that were
enrolled at a control at a different time.

It's exactly the same scenario where
it's only time that's different

in the scenario to make this.

Don Berry: So we, we put it in the trial.

Uh, we're going to use the,
uh, all of the controls.

Uh, and there is this, uh, time
machine that's Scott, uh, talked about.

And in the example I gave in the
previous and part A of Pertuzumab, uh.

Pertuzumab the control arm At the
time that we did, uh, the analysis

and graduated, uh, the control arm had
about a 20% in triple negative breast

cancer had about a 20% path CR rate.

And the, in, on the basis of the one
patient that we had from, uh, Pertuzumab

who had the results of surgery.

The result of surgery was A-A-P-C-R
and these other controls, uh, these

other, uh, patients, um, that we use
the MRI to predict, predict and to

conclude that the result was going to
be, you know, 60% for, uh, Pertuzumab.

Um, we very little information.

About the experimental arm, of course,
but that information was so compelling

that it was to the right of 0.2,

uh, with high probability.

And so it met the graduation
threshold, which was a predictive

probability calculation.

So, and, but we didn't stop there.

We did it for, for all of I spy two.

From that point forward, all of I spy
two was based on the time machine and

this, uh, the FDA calls them contemporary
controls, controls that were in the

trial, but not at the same time.

Um.

And they've agreed to use that approach.

It's really

amazing, uh, the time machine
approach in registration trials.

So the two trials that I mentioned,
one in, uh, glioblastoma, the other

one in pancreatic cancer, uh, they
use the, the time machine as a

fundamental thing from the get go.

It isn't just when you had to stop
the, the experimental, the, uh,

control arm that this applies,
but it applies for everything.

And

as

Scott mentioned, you know, uh,
uh, Ted Williams in his list was

way back in the, in the middle.

Um, we.

Uh, we want to use, um, uh, Ted
Williams during the that period.

Uh, and,

uh, we want to use an experimental arm.

We want to compare Ted Williams.

We get, we get a notion of what is the
control rate during, uh, Jimmy Fox's

period during, uh, Hank Aaron's period.

Uh, and so we in the algorithm explicitly
compare Ted Williams to Jimmy Fox.

We explicitly compare
pertuzumab to other therapies.

It's just we don't tell.

Merck as the, uh, the drug's,
uh, uh, sponsor owner.

Uh, we don't tell Merck what
the Eli Lilly drug was doing.

We don't tell Eli Lilly what the Merck
Drug is doing, but we're comparing

everything to everything in order to,
uh, uh, assess what is the control rate.

Scott Berry: control.

So, uh, behind the scenes there's one
model that's estimating the individual

interventions in the trial and provides
that estimate to a standardized control.

And it's using all of the
data from all the arms.

'cause it enables us to estimate
the effect of subtypes, the effect

of time, all of these variables
in providing that estimate.

Then you can hand a particular
sponsor the estimate for

their arm relative to control.

And the machine is used, all of the data.

Now you, you mentioned
the, um, graduation.

Uh, which by the way I think is a
new word that came out of ICE by

two as sort of some level of success
that wasn't statistical significance

traditionally, but the success in ICE
by Two was innovative in and of itself.

What was graduation or success
from a phase two trial?

Don Berry: So the way we defined
graduation was it's going to do well.

Um.

And we did predictive probabilities, which

is kind of essential in this business.

That you, you, and integral to the kinds
of things that Barry does, um, we look

at the data, well, we, I say we look
at the data, the algorithm looks at the

data programmed to look at the data.

It's kinda like, uh,
artificial intelligence, um,

um, and, and learns, uh, what

data.

Uh, what, what are the

Scott Berry: the

Don Berry: current probabilities of
benefit for the various therapies?

And we do prediction.

We say, well, is this, does the data
at hand suggest that this is a drug?

That this is a drug that can be
marketed, that's gonna be successful

in the context of ISI two?

Is it gonna be successful
in a phase three trial?

And in a phase three trial predicting
the results of a phase three trial?

Uh, the Bayesian approach, of
course, is critical in this.

Uh, the,

uh, PCR rate in the experimental drug
has a probability distribution called

a current probability distribution or
posterior probability distribution.

Uh, but what really matters is the future.

So there's data, there's
two kinds of uncertainty.

One is in what is the PCR rate,
that parameter, uh, and the other

is what are the data that come
forward, the inevitable uncertainty

associated with the future results.

So these two uncertainties combine to
allow for calculating a prediction.

What is the probability that a
future trial will be successful?

If it has these characteristics,
and that drives everything in.

I spy too.

It drives the adaptive randomization,
it drives the graduation, it drives the

possibility of stopping for futility.

There's a stopping for futility.

Um, that's across the board that we
calculate based on the predictive

probability, uh, distribution.

But there's also, as Scott suggested, the,
the possibility of stopping in a subtype.

Um, because the randomization
probability is so small that it, the

patient, you know, no patients are
gonna be assigned to this subtype.

That actually happened.

Neratinib was one of the
early drugs in the trial.

We published them in New England Journal
of Medicine, the results of the Neratinib,

uh, trial and indicated that there was.

Some subtypes where it got zeroed out,

Scott Berry: out,

Don Berry: but meanwhile, in other
subtypes, it was doing very well.

So it stopped randomizing in the
subtypes where it was not doing

well and increased the probability.

And the other subtypes graduated since,
uh, has been approved by the FDA for

the, you know, marketing, uh, approval.

Um, and nobody, as Scott indicated,
nobody knew that this happened.

I mean, if this happened in a two arm
trial and you, you said, uh, the DSMB

tells you, you gotta stop the control,

Scott Berry: control,

Don Berry: that would be a big deal.

How could you stop the control?

Could you then go in and, uh, get
another control or, or wait a while

until you figure out what to do?

We never announced that Neratinib was no
longer being assigned to those subtypes

of, of women, um, until after the trial.

Of course, the, the, uh, DSMB
was involved, uh, and the DSMB

was enthusiastic about doing it.

In fact, before they realized that it
was going, that it wasn't assigning

probabilities, uh, wasn't assigning,
uh, neratinib to any of these subtypes.

Uh, one of the DSMB members said, we
should stop Neratinib in that subtype.

I said, you don't have to.

algorithm has already stopped it.

Scott Berry: Yeah.

Yep.

Okay.

So a little bit of the numbers of I Spy.

So what are we talking about?

I SPY two started rolling in 2010
and it ran through 2022, and I

believe in that time, 23 different
investigational therapies went through

Don Berry: Correct,

Scott Berry: and nine of those
were successfully graduated.

So you talked about pancreatic
cancer, you talked about GBM where.

Very little works.

And it's the, the huge challenge
of these platform trials is

to, to try as much as you can.

And breast cancer therapies worked
and some of them in subsets of women,

the right subsets of women, but
nine of 23 successfully graduated.

And, uh, I, I think you mentioned
that four of the nine have

received marketing approval.

Don Berry: Yeah, something like that.

But there's still, the others are still

Scott Berry: To be

Don Berry: Yeah, no.

Scott Berry: Yeah.

Yeah.

Yeah.

So from the perspective of
the impact on breast cancer,

the I-SPY 2 trial was amazing.

Before I sort of turn to the, the, the,
the larger impact of I Spy two and other

things, I, I think there's a few people
that you wanted to make sure that due

credit, one of them is Janet Woodcock.

Don Berry: Yes.

So this is really a story
about three dynamic women.

Uh, one of course is Laura Esserman.

Um, uh, the other I mentioned is Anna
Barker, uh, who was, uh, very helpful in

the, the funding aspect and then give and
giving advice about how, how to do this.

Uh, and the other is Janet Woodcock,
who was at the time the, um, head of

CDER uh, at the FDA and, um, has been.

Uh, pushing adaptive designs and, for
example, this thing about adaptive

randomization, she is, um, publicly
announced adaptive randomization

is adequate and well controlled.

Now, if you know

regulatory stuff, you know
that that's, that's the

criterion for a phase 3 trial.

For a pivotal trial, for a trial
that, uh, a registrational trial,

adequate and well controlled.

So she has been enormously helpful,
um, in I-SPY 2 and in setting

up, uh, GBM Agile and uh, uh,
the pancreatic cancer, uh, trial.

Scott Berry: so, uh, Anna
Barker, Laura Esserman.

Janet Woodcock, a huge role
and a couple people, perhaps a

little bit behind the scenes.

So Meredith Buxton was there for quite a
while, had a huge impact on ice by two.

Don Berry: And the current
circumstance with Meredith Buxton is.

Scott Berry: Well, she is now,
she's graduated, you will.

Um, uh, she successfully, so she now
runs, uh, uh, she's now the CEO of GCAR,

global Coalition for Adaptive Research,
and they run multiple platform trials.

Uh, so she, she did, uh, incredible work
on a lot of the logistics and making,

making this go and now does this for GCAR.

And, uh, a little bit on a personal
side from Barry Consultant's side is.

Berry Consultants built the code to
run Ice Spy Two, and Ashish Sunil

at Berry was intricately involved in
making sure that code ran, working

with Meredith running for years.

Uh, Ashish, Sunil had a huge
impact on making that trial run.

He has sort of, uh, uh, very, very
sadly, uh, came down with a LS,

uh, and, and uh, in that, but he
had a huge impact on ice by two.

Don Berry: Uh, so I should
mention, um, Kyle Waltham.

Kyle was at, uh, MD Anderson with me when
we set this up, and he wrote the original

code, um, and it, it worked great.

Um, and then when we had to vary
from the code for, you know,

we're adaptive, um, we went back
to MD Anderson and, uh, they, we.

I had lost the code.

So I went to Scott and I said,

what can I do?

Um, and he said, well,
I'll, I'll rewrite it now.

You have to understand the original
code was really long and, and

developing and, and writing and deciding
what we wanted to do and all this

adaptive randomization and et cetera.

Um, and it took and including
with some funding from Anna

Barker, um, for Kyle's salary.

I never, uh, had any kind of a of a.

Uh, salary or anything.

It was all voluntary and all of
Barry consultants' work was, was

voluntary, uh, in ice py too.

Um, so Scott took, I think three days
instead of, instead of, uh, two years,

it took Scott three days, but of course
he knew what we, what the goal was.

Scott Berry: Yep,

Don Berry: Uh, and that's,
that's what we've used since and

model modified it, uh, since I.

Scott Berry: Yep.

So I-SPY 2 in many ways was the,
uh, a forefather a a a foremother

for, for platform trials, one of
the, the early platform trials.

Phenomenal success, success in
the disease, but is really in

many ways changed the industry.

GBM Agile doesn't exist.

Pancreatic Cancer Action Networks
trial PanCan, that, that, that trial

Precision Promise doesn't exist
and COVID hits and, um, operation

Warp speed, the ACTIV trials.

Are largely I-SPY 2 for treatment
of therapeutic COVID and Janet

Woodcock's leading the ACTIV network
and it's all platform trials.

So the impact of I-SPY 2 and this idea
that you said, you, you bet the farm at

the time, and you and Laura kind of went
on this strange, uh, risky path that

cooperative groups might not embrace.

And you get, you know, you get innovative
funding, you're getting Safeway

grocery stores to help fund this.

Um, and it's, it changed
the world literally.

Don Berry: Thank you, and I agree.

I

Scott Berry: Yep.

Alright.

So they're sort of b before I
spy two and after I spy two.

So, uh, phenomenal trial, uh, phenomenal
effort and, and an amazing story.

So Don, I appreciate you
walking us through that story.

Don Berry: thank you for

leading the walk.

Scott Berry: walk.

Yeah, right.

Well, thank you everybody for, for
doing two parts of I Spy two and in

the next time we are in the interim.

Thank you.

Don Berry: Thank you.

Scott Berry: Thank you.