Why Scan for Leaked Secrets
When you're using AI to code, whether it’s Lovable, Replit, or Claude Code or Codex on a local IDE (or really any other AI coding tool), you know that it's easy to create something quickly and make changes rapidly.
But a lot of times, AI skips the important things like properly checking the security and making sure that unsecured information isn't getting out all over the internet; information like API keys and passwords that might be hardcoded, database URLs, all sorts of private data AI is just throwing out there and making available to those who know where to look.
You want to know about these security leaks in your app before somebody else figures it out.
This article is going to be the first in a series where we show you how to check different security aspects of your AI coded applications.
This isn't a full security check. This is one piece of several things you'll need to do, and it's a really good starting point.
We're going to use our Betterleaks scanner.
Introducing the Betterleaks Scanner
Betterleaks is an open-source scanner (built here at Opichi and something we use ourselves) that looks for things in the repo of your code that could expose something important in your application that you don't want others to have access to.
We’ll walk through how to use the Betterleaks scanner here, but if you want to follow along with a video tutorial, scroll to the bottom of this article and click through to our full YouTube video.
If you have an application on something like Replit or Lovable, you're going to want to first go to our YouTube channel and pick one of these videos that shows you how to get the application downloaded onto your computer, and follow that process.
How To Move Off Lovable Cloud: Complete Railway Migration & Deployment Guide
Move Your Replit App to Railway
Lovable to Local with Claude Code
Stop Wasting Replit Credits: Use Claude Code
If you're using something like Codex or Claude Code, you probably already have the application downloaded.
Next, we're going to go through the process of checking this first step of security.
Get the Betterleaks Prompt from Opichi
Go to opichi.com/resources to find our Betterleaks and click the ‘Get the Prompt’ button. It's going to take you to download the prompt. The prompt will drive this whole process for you.
Setting Up Your App Scan in Codex
We use Homebrew as the installer. If you've been doing AI coding, you know by now that most computers already have Homebrew installed.
If you don't have Homebrew installed, you can copy https://brew.sh/ into your AI tool of choice, and it will walk you through how to install the open-source package manager.
On our desktop, we're using VS Code for this walk through. It's free from Visual Studios and runs Codex and Claude Code agents, so it's great to help keep your costs controlled.
You're going to copy the Betterleaks prompt and open up your app code.
Running the Scan in Codex
Now we paste in that Betterleaks prompt into our local IDE. In our video tutorial, we're using Codex set to Sol, Extra High effort.
Codex is going to check if Betterleaks scanner is installed. If not, Codex is going to check the installation instructions to see how to install it, and will then run the install. Then it's going to use that to follow all the steps in this Betterleaks prompt to go through and check for secrets and information that might be leaked from this application.
Once it’s installed, the Codex is going to start the rest of the scan. Depending on the size of your application, this could take a while. It might be a few minutes; it might be 30 to 40 minutes. Let it comb through this scan and it will return its findings back to you.
Running the Scan in Claude Code
While that is running, we want to switch over to Claude Code and run the exact same scan, just to see how the results compare.
We're going with Opus 5. At the time of this tutorial, Opus 5 is a recent model.
We're going to paste in the same prompt.
Its feedback is a little different from Codex, but it's going through to do the exact same thing.
Running Claude Code, it found the same five things. This is because Betterleaks is a deterministic tool. That means it's actually using code to find some of these issues, it’s not an AI-only tool. We're pairing this deterministic code with AI, either Codex or Claude Code, to get better outcomes and understand the meaning of what is found.
Comparing Claude vs Codex Findings
Now that both scans are done, let's compare:
Both found 5 things to check, but they took a couple different routes explaining these issues.
In Claude's finding list, it had two actionable items. Claude Code said that the backup password for user login is written into the code and it gave instructions for what to do.
It also found that customer AI keys are stored unprotected. The AI told us they're encrypted, but the Betterleaks scanner exposed they're actually stored in plain text.
Obviously, we would consider both of these to be important things to fix.
Let's take a look at what Codex found…
Codex reported back four findings that are actionable items, not two.
The first one states stored customer credentials are sent to browsers. User responses expose configured Gemini keys while the admin response exposes every password hash and configured Gemini key.
We would consider that to be a pretty big security gap and that should be something that we fix right away.
It also found the server accepts a predictable fallback session key. This is the same thing that was flagged in Claude Code, so we should fix that.
Codex also said the Gemini keys lack application level encryption. That's the same thing that was flagged by Claude Code, too. Get that on the plan-to-fix list, too.
Handling False Positives
Finally, Codex reported that local secrets are synchronized on Google Drive. This is a false positive. It thinks this is in a Google Drive folder, but it is not. It used to be a Google Drive folder, but it isn't anymore. The secrets definitely aren't synced here.
This is an example of one of those things that it might find as a false positive, and you need to look through it to confirm that’s exactly what it is. It's way better to be safe than sorry.
If nothing else, consider it an exercise in deeply familiarizing yourself with your code.
Fixing Issues with Plan Mode
The next step in either the Claude Code or Codex situation would be to go through and talk with AI about fixing each one of these findings. Switch into Plan mode and ask it to make a plan to fix each of these issues that it found.
You can tell it to ignore something like a false positive if you know for sure that's not an issue.
Recap & Getting Help
Your application will look entirely different from our video tutorial, but this is the process. So, to recap:
You're going to get the app down on your computer if you don't have it already. Again, you can use our YouTube channel videos that show many different ways to do that.
Once you have that on your computer, go over to opichi.com/resources. Get the prompt.
Once you have the prompt downloaded, you're going to open the app on your local IDE (we used VS Code to run both Agents), and you’re going to use that prompt to run this scan and get you to this point where you can start to fix some of these security issues.
If you get stuck fixing any of the security risks or you're not sure what the AI means, you can head over to opichi.com/contact and grab a 15-minute review. We'll take a look at it with you and see if there's a way we can help you solve the security issues that come up.
And again, remember: this is not a full security solution. This is one step in the process of securing your AI coded app, but it's a very important one and it's a really good place to start.
If you have any questions at all, drop them in the comment section of our YouTube video or go ahead and book a time to chat with Courtney and the team.
Watch the Full Tutorial on YouTube: https://youtu.be/n9eyZ7ZW8rA?si=KoSXtZZ2XoShVTiY