I Tested AI Cloning… and It’s Way Worse Than You Think!（原声文稿）

Hey, I'm Dave, welcome to my shop. I'm Dave Plummer, retired software engineer.

from Microsoft, going back to the MS -DOS and Windows 95 days.

And by now you've likely heard the news of someone cloning the entire Photoshop. suite. And then the entire Microsoft Office suite.

Just as they dreamed for nuclear power back in the 50, software is now too cheap.

to meter. Heck, it's free now, right? Well, I'm going to do two things today.

I'm going to give you my hot take on whether AI cloning of applications is.

ethical and legal. and thanks to a little help from the NSA, I will then show you. precisely how it is done. And it all starts with a simple question.

Can I give an AI a finished Windows application without supplying any of its.

source code or symbols and have the AI produce the exact same application but. written completely anew in Rust?

And so can you take the same controls, calculations, and actual behavior all the.

way from somebody else's executable to a new executable that does the exact same.

That's the experiment today. And if it works, there's a second question waiting. behind that. Who actually gets credit for creating what?

This morning I started with Microsoft's calculator, calc .exe.

I wanted to feed the program into reverse engineering tool, have the AI reconstruct.

its behavior based on the code, and then compile that reconstruction into a working. Rust application.

Now of all the Windows applications in the world, why pick on Calculator?

Well for one, it's a rare piece of Windows that has been placed under the MIT license.

so maybe the jack -booted thugs won't kick in my door for messing with it.

And the other reason is that I used to own Kelk when I worked at Microsoft. I didn't.

write any of the cool code in it, I was just the dev manager overseeing the. developers doing the real work.

But believe it or not, the calculator is very complex internally. It has arithmetic.

and programming and accounting modes and works in multiple number bases and has. infinite precision mathematics routines.

There's a lot going on under the surface, stuff that I was confident the AI could. not deduce from inspection or by reading about it elsewhere.

Instead, it would have to actually understand the code in the calculator. binary to make any real progress.

More important, I don't think you could deduce all of Kelk's sometimes quirky.

behaviors purely by inspecting the running application. If it does it, I think it's. proof that it's read the recipe.

And that can matter when a perfect reproduction is the goal. It has to be.

really identical to the operation of the original. And the reason this might work. is hiding in plain sight.

An executable contains instructions. Every decision the program makes still has to. happen somewhere in some code that somebody wrote.

That gives us something to investigate even after the original source file is no.

longer available. The tool I used was Gidra, the reverse engineering framework. released by the NSA.

It can disassemble machine instructions and construct a higher -level, C -like. description of their behavior.

The description is an analysis of the compiled program rather than a restored.

copy of the original source code, because it doesn't know any of the names or the. symbols or anything.

It's just working backwards from the binary logic and it kicks out C code that.

would compile to that logic, but the names are going to be best guesses.

Internally, Gidrick can translate processor instructions into simpler.

operations called P code. That's actually how many of the old Microsoft applications. for Office were written.

Micro instructions instead of assembly language, which was then assembled or. compiled depending on how you look at it, into machine code.

And so, P code operations describe things like loading a value, doing arithmetic, to. extorting a result and choosing which path to follow.

It gives the analysis a common vocabulary for what the processor is doing.

The particular intermediate file is in our experiment matter, but the essential route.

is executable, analysis of its operations, reconstructed rest, and then another. executable.

Imagine the calculator receives a digit. Somewhere it has to decide that that digit.

extends the number already on the display, or whether it starts a new number that. you're typing in.

Somewhere it remembers whether an operator is pending. After you press equals, the. next digit may need to begin a fresh calculation.

Those are the state transitions, and the compiled program contains the machinery.

that makes them happen. The AI's job is to follow that machinery and express an. equivalent set of decisions in Rust.

A load from memory might become a field in a structure. A tangle of branches might. become a match expression.

The resulting source can look quite different while preserving the behavior of. the code that supplies the evidence.

Now, like I said, you're not going to get any of the variable names and none the.

comments and the other niceties that do not survive compilation, but you do get a. working, if somewhat unreadable, source code project.

Of course, arithmetic is only part of a Windows application.

The executable can contain resources, such as dialogue layouts, strings, icons, and.

so on, while imports tell us about what calls are being made into other. components.

A convincing reconstruction has to account for the interface as well as the.

calculation engine. Getting a summary doesn't tell us whether the keyboard.

shortcuts work, or whether the clear button resets to the right state.

A calculator is a particularly good piece of software making a superficial clone. look respectable.

Most of us can erase the buttons in some kind of editor, and most of us can. implement addition and subtraction and so on.

The difficult part is reproducing the accumulated decisions behind those.

buttons. And so it has several opportunities to fail. The analysis might. misunderstand the function.

The rest might compile but implement the wrong state changes. The interface might.

resemble the original while quietly behaving like an entirely different. calculator.

I'm going to show you the actual process now. Watch how the executable becomes.

information that the AI can work with and how that information becomes the source. code that a Rust compiler will accept.

Then we'll launch the result and compare it with the original. But first, speaking. of tools that show you exactly what's happening under your hood.

if your computer's fan just spun up or a frame dropped or the battery started.

draining for no obvious reason, most task managers will show you a number that looks. okay after the fact.

30 years after writing the original Windows Task Manager, I built a new Task.

Manager OG to name the process while it's still happening. And it's tight native. code all the way down.

Win32 on Windows, Swift and AppKit on the Mac, QT on Linux, all over one shared C++.

measuring core. There's no electron, no browser pretending to be a system tool.

The free version gives you the important live view that you need. Pro adds the.

flight recorder so you can actually record the slowdown, scrub back through it and. see what the CPU.

GPU or memory and so on did to cause the actual damage after the spike is gone.

It's at tmog .org if you want the free download for Windows, Mac and Linux.

Alright, let's make a calculator. Here is my prompt. Okay, your mission is to. rewrite calc and rust.

which we are going to do by decompiling the binary to an intermediate format of.

your choosing, then translating the logic completely and accurately to Rust.

then compiling that Rust code to produce a new binary called AI -Kelk that should be.

functionally equivalent to the original Kelk which was likely authored in C.

So this is a Cedar Rust translation project without access to the source code.

I want you to iterate on the project until all of your tests pass and the resulting.

binary executes and operates in the same way as the original Kelk, looks pixel. -perfect, etc.

Now it's going to ask me if I want to do the entire application or just the.

launcher And of course we want to do the entire application, which means all of its. dependent DLLs that it needs as well.

Now we'll just skim through some of its progress. You can see it's used Ghidra and.

it's recovered 10 ,195 functions in the calculation DLL, which sounds about right.

Next I believe it's bragging about its 131 ,077 command classifications and the 8201. dispatch tests that it wrote.

It also tried 2 plus 3 equals 5 without opening the UI. Now that's reassuring.

It also says the Gidra has finished analyzing the UIDLL exports, recovering 18.

,959 functions with no decompilation failures.

Now it has tried something like 357 different command sequences and 8 ,032.

different intermediate displays and made sure that they match the original engine. exactly.

Now, in its own testing, it has found the number of errors which it has then gone. back and fixed the code to make sure that it passes the test cases.

but we should We take that as an indication that it is not a perfect. deterministic clone of the original logic.

It's obviously making some kind of assumptions along the way, and it's.

filling in some blanks and it's doing it wrong because it's noticing the outcome is. wrong.

So the fact that it has to go back and reinterpret or repair the original code.

means we really can't assume that there's anything fully accurate about this clone. that it's building.

It might be close, it might be near perfect, but it's not going to be perfect.

And with that, I'm just going to fast forward through the test phase and let's. see what it actually produced for an application.

All right, it walks like a calculator and it talks like a calculator, but does it. calculate like a calculator? Let's give it a shot.

We'll do 1, 2, 3, plus 4, 5, 6. I'll have to get my other calculator out to verify. if this is correct.

But pending verification, it looks correct to me. So, now we have a working Rust.

application and it's a close approximation of Calculator.

Now we just have to talk about what we made. We recreated Calculator's appearance. and functionality in the demonstration that you just watched.

I asked for the job in one shot, but that describes my interaction with the AI. It.

doesn't mean an automated workflow has to accomplish every internal step without. checking or revising anything.

And I want to be precise about the word perfectly because programmers have a habit.

of making us regret that word. What we've demonstrated is a working reconstruction.

claiming complete equivalence to require a much broader comparison, including obscure.

input sequences and behaviors that we have not yet exercised here.

There could still be a discrepancy waiting behind a key combination that nobody has. tried since 1997. As said, however.

the AI also produced a comprehensive test suite that runs tens of thousands of.

possible types of computations through the calculator to make sure it always does. exactly the same thing as the original.

But, then again, Intel probably had a pretty robust set of math tests for the.

Pentium 2, so there really are no guarantees. That distinction doesn't make. the result ordinary.

The AI crossed from a finished application to newly generated source in another.

language and then back to an application. The original source files weren't required. as an input.

The executable itself supplied a root into their implementation.

Now, I spent my career working on Windows Software, so I have a fairly strong.

attachment to the idea that writing an application involves writing the. application.

Whereas here, I was proposing to start at the other end of the assembly line and ask. the machine to work its way backwards.

which sounds like a useful recovery technique right up until the application.

you're recovering belongs to somebody else. To understand what happened, follow. an ordinary program on its trip out the door. A developer writes source code.

The compiler turns that source into machine code and the linker brings the.

pieces together into an executable, with their appropriate references to libraries. and operating system services and so on.

There are plenty of details in between, but that's the journey that matters here.

During that journey, a lot gets thrown away. Your helpful variable names. disappear.

your comments certainly aren't required to run the program, and a function that you.

spent the day trying to carefully separate from another function might end up getting. inlined into it anyway.

Several different ways of writing an operation can end up producing the same.

instructions after optimization. Consequently, reversing that process. doesn't deliver the original source.

If somebody compiles a program containing a variable called total, there isn't.

necessarily a secret total hidden inside the executable waiting to be unzipped.

The compiler kept what the computer needed, which rarely includes an.

explanation of what the developer was thinking after the third cup of coffee.

But we don't need to recover the developer's original sentences to recover.

some of what those sentences made the computer do. And that's the opening that. makes this experiment interesting.

And there's our first important realization. Different source texts can. describe the same computation.

You can change the language, rename every variable, and reorganize the functions.

while keeping a very substantial connection to the program that you started. with.

Seeing Rust in the editor doesn't tell us how independently the program was.

designed. Which brings us back to that phrase that initially made this sound. reassuring, where you're using none of the original code. But what do we mean by code?

If we mean the original source files, I don't have those, didn't hand them over.

and I later audited the code for plagiarism and made sure that it wasn't. using external code as samples, just in case.

But an executable is compiled code. This assembling it doesn't turn it into an.

unrelated set of facts, and translating an analysis into Rust doesn't erase where the. analysis came from.

We need to distinguish new source texts from independent authorship. This is the.

point where my reaction changes. As an engineering demonstration, this whole. thing blows my mind.

As a proposed way to take somebody else's proprietary product and manufacture an. unauthorized substitute, I think you should be illegal.

Depending on what has been copied and how, existing law may already make it so. My.

instinct is to call that piracy and changing the programming language doesn't. improve my opinion of it.

That's just my ethical judgment, because I'm not a lawyer and my free opinions are. priced accordingly.

But the legal answer has a complication that we need to take seriously, especially. if we're going to make a strong argument.

American copyright law doesn't grant ownership of every function that a program. performs.

Section 102 of the Copyright Act excludes ideas, processes, and methods of operation. from copyright protection.

Microsoft doesn't get to own addition because calculator adds numbers together.

Someone else is allowed to write a calculator, including a very good one. without asking Microsoft's permission to calculate.

The code expressing these functions can be protected as can other original. expression. protection.

Some protection can extend beyond literal source text, which is why changing every. variable name isn't a universal escape hatch.

Legal analysis has to separate protectable expression from functional requirements. and other elements that copyright doesn't cover.

That could be a difficult exercise in a program both almost entirely to accomplish.

a task. The analogy I keep coming back to though is translation.

Translate a novel into another language and you get new words, but you haven't. thereby become the original author.

You can't claim it as your own work, because it relies on the creative input of. another person to have written the original.

Software makes the analogy a little more complicated because copyright doesn't own. the underlying computation.

Still, a new language by itself tells us very little about whether protected.

expressions survive that trip. And courts have already allowed some reverse. engineering.

In Sega vs. Acolyte, the court allowed disassembly under the circumstances of.

that case to reach unprotected functional elements needed for compatibility.

In Sony vs. Connectix, intermediate copying of the PlayStation BIOS during.

development of the emulator was held to be fair use.

Using the original program during development did not automatically settle.

the case against the developer. Now, those decisions don't issue a general permission. slip to duplicate commercial applications.

Sega expressly left open infringement claims about the finished products. They.

do mean that whether a binary entered the process isn't a complete statement of. American law, no matter how I feel.

There are excellent reasons to investigate binaries from analyzing malware to build.

incompatible replacements. I spent enough time keeping old computers alive to. appreciate that.

Decades ago, a company called Phoenix made their own version of the IBM PC BIOS.

basically a work alike so that clone manufacturers could build PCs around them.

IBM was, of course, outraged and sued. But Phoenix had not copied IBM's work. What.

they had done instead was known as clean room engineering.

or white room, and where one engineer documents the API without ever looking at.

the code that implements it, and then another engineer or team of engineers. creates an API precisely to that specification.

Done correctly, it should be a perfect replacement even if implemented completely. differently.

And the court ultimately ruled in Phoenix's favor in large part because the.

API itself was public and none of IBM's actual implementation was involved.

But the original PC BIOS contains only about 12 entry points, I think it is. An.

application like Photoshop has thousands of interface points.

You certainly could clean room Photoshop, but the work just in producing the spec. would be prohibitively large.

One ostensible way around it would be to use automation to drive the original.

application in real time and figure out everything there is to know about it and. then build from that.

I don't think that's quite possible yet, but one day, of course, it will be. There's another wrinkle with this particular example.

Microsoft published the modern Windows Calculator source code under the MIT. license in 2019.

That doesn't establish the license of every historical calc .exe, and the.

repository excludes certain components and data. Not supplying source files is. something we can establish about this experiment.

Proving that a model has never encountered related public source during training is a.

very different proposition, and I asked it, and it assured me, but you never know.

The footage shows the workflow that we used. It doesn't give us a complete.

biography of what the model was doing behind the scenes. But the broader. question survives those qualifications.

Suppose the input is a proprietary application whose owner has not authorized.

a rewrite. Suppose the goal is to reproduce its distinctive implementation. and appearance closely enough to replace it.

Should we be able to feed that product into an automated translator, change its. language, and declare the result new and independent work?

I think that deserves considerably more scrutiny than it's currently receiving. This matters to me personally because of Task Manager Tmog.

When I used AI to help build that, I specifically did not feed an existing. application into the process and then did that for a translation.

Instead, I wrote a 107 -page specification describing exactly what I wanted, and then.

it took another 400 -500 GitHub PRs to get it to that point.

That meant making decisions about the product and expressing those decisions. before asking the AI to implement them.

A specification isn't automatically innocent, either, because you could put.

somebody else has protected expression into a specification and used the extra. step to disguise the copying.

Even starting from a prompt isn't proof of legality.

but there's a meaningful difference between designing the behavior you want.

and using another application's implementation as the raw material for. your own.

That's the distinction that I want us to preserve. When the route is my.

requirements, generated code, compiled application, I can describe the AIs. helping me implement my design.

But when the root starts with an existing binary, proceeds through analysis of that. binary and ends with a replacement of that binary.

I want the result identified as a clone derived from that application. Permission.

may make it perfectly legitimate. It still has an ancestor.

My objection is to using the translation step to claim the original work. contributed nothing.

If your process depends on somebody else's application or creative input of any kind. acknowledge and embrace that dependency.

And if you're reproducing protected expression, calling the output rust.

doesn't supply permission. A compiler produces an executable.

It does not produce a certificate of independent authorship. Now extend the.

Thought Experiment to Photoshop, or taking final cut from the Mac and producing a. Windows version.

Those are much bigger engineering problems, at least to get them right and.

have them work. And if you're curious, the big replacements tend not to work very. well from what I've seen.

Because a large application depends on libraries, assets, file formats, plugins.

and operating system services that won't obligingly turn into portable rust simply.

because they're asked nicely. Calculator doesn't demonstrate that we can port Final. Cut tomorrow.

Now, any timetable I give you would be speculation. What it does demonstrate is. why we should pay attention to the direction of travel.

Reverse engineering has traditionally demanded a great deal of specialized work.

AI can assist with interpreting the evidence and turning it into another. implementation.

If that becomes faster and more reliable, the economics of making an authorized.

substitute could change even while the legal principles remain familiar.

And there are valuable possibilities here too. A developer who owns an application.

could use these techniques to recover lost work or help migrate it.

With permission, reconstructing old software in a maintainable language could. be a terrific tool.

The same capability becomes much less appealing when the application's owner is. simply treated as an inconvenient detail.

We began by asking whether an AI could work backward from a Windows executable. and forward again into a Rust application.

In this experiment, it produced a working reconstruction. The answer about perfect. equivalence still depends on verification.

and the answer about legality depends on more than the language that we compiled it.

My view is that feeding somebody's proprietary application into an automated.

rewrite to manufacture non -authorized substitutes should be illegal.

Existing law may already prohibit particular cases. I won't pretend the.

binary alone decides that. But a new language has not persuaded me that the. original work contributed nothing to the process.

And now that you've watched the process and have a better understanding of it. consider how you'd feel if the application going through it were yours.

If you found today's episode interesting or entertaining, remember I'm mostly in.

this for the subs and likes, so I'd be honored if you would consider leaving me. one of each before you go today.