I wanted to share a mini side-project I built this month as it gave me many new insights on AI, where it is going, and where/how it will succeed.

While I normally focus on product, strategy, and company building (I am currently working on Second Coffee, an AI-first incubator), I still love to code. :)

So, What is CSS Genie?

CSS Genie is a Chrome Plugin that lets you select parts of HTML pages, ask for edits in English (like "make this a more modern square look" or "Change the color scheme so it is more friendly"), and the Genie will a) update the page in real-time, and b) tell you which changes you need to make to HTML elements to make it happen.

Why Would CSS Genie be a big deal?

I have generally found developers (with some exceptions) often divide themselves into frontend and backend developers. While frontend developers enjoy spending time making the perfect UX and UI interfaces, backend developers tend to dislike spending hours getting the pixels lined up.

CSS Genie is an attempt to give backend developers frontend UI superpowers, allowing them to make beautiful frontend updates to their code by looking at the current page and giving English instructions.

Current Status

Right now, CSS Genie is fun demo-ware. It works about 60% of the time and takes about 1-2 seconds to run. It is much better than it was when I started (30-40 seconds to run and 20% successful), but IMHO still too inaccurate for production. It also ended up being more cumbersome compared to other solutions. I have used it a few times for other projects, but I am not sure as it stands it will be a long-term tool.

Technical Details

For those interested, the technical stack for CSS Genie includes:

• Chrome Extension (Javascript) using browser Sidebar.
• LLM API Server Proxy: A Phoenix/Elixir server to handle CORS issues and optimize HTML input for token efficiency.
• Prompt/Response pipeline: Transformations of the input and output in Elixir to compress the token size by 35% and increase accuracy as well as cleanup the usually poor JSON output and reduce security risks.
• GPT APIs: Tested OpenAI's GPT-4, GPT-3.5, a local LLama 3, and eventually settled on Groq
• Prompts: Multiple iterations to ensure consistent responses.
• What I would build next:
  • Multi-parallel / sequential pipelines for accuracy. It would not be too hard to make 3+ parallel requests to LLMs, and have another LLM trained to pick the best of the options, or run one LLM for the code and another that is really good at editing/refactoring. The main limitation right now to these is cost & speed but that should become more viable over time. 
  • Security patches - The current version leaves open a few security holes that I would need to patch before any productization
  • Improved prompting and examples

Key Learnings:

While CSS Genie is not going to be productized, there are definitely a couple of major key insights.

Do Not Underestimate Speed

How quickly a GPT responds matters SO much more than we think. It's hard to know how much of a game-changer speed is until you experience it for a use case that was unusable at slower speeds. 

When I started building this extension, I first used GPT3.5 and GPT4. The response rate was 15 to 30 seconds which made the product unusable (and you could not hide it by streaming). It was only after I was added Groq’s LLama3 that the product became functional. (I love Groq.com but they need to open up their throttling :))

The experience made me realize that speed is very much limiting the types of applications that can be built in AI. Right now, only a small percentage of use cases like chat interfaces with streaming results or asynchronous activities - can work with today's technology.

Increasing the speed of GPT will unlock an order of magnitude of new uses such as auto-realtime categorizing, speedy LLM-based sequential pipelines, backend smarts that happen when you interact with a website, realtime voice interfaces that have no lag, having multi-layered GPTs, real-time agent GPTs, multiple parallel processing etc....

In many ways, a 10x faster GPT may be as important than a 10x smarter one as it will allow really interesting backend interfaces that we just cannot do right now.

AI in Production - The Devil Is Still in the Details

As I have seen from all of the AI projects I have been involved with - an AI-first note-taking app, Chat with your NFT, GPT chat analytics tools - AI in production is still far harder than just plopping in the OpenAI API. You need to have to processes, tooling and code and a team to constantly figure out how to improve accuracy and speed, and overcome a number of security, scalability, pipeline, and other challenges. The space is still nascent so make sure you have the right team to put it all together and have it be reliable and scalable. It does get a bit easier though when you have experienced the problems again and again :).

CSS Genie was fun but not as good as my IDE (Cursor).

While I thought would prefer an AI CSS editor embedded in my browser, it turns out I prefer it in an IDE. Even though the web version was faster and I could see the UI change, I still needed to manually find the appropriate sections of code and copy the changes - which was not necessary in the IDE.

We have just scratched the surface on AI - and we are still early.

The experience reminds me again we are just scratching the surface on what new AI tools can do.  LLM technology is improving so quickly we are only a year or two away from a whole new set of problems getting solved by AI based products.  I am even more convinced we are in the same type of situation as the late 1990’s internet.  AI is hyped but it is good hype as a number of the reasons it is not working are technical. Once those barriers are removed, the sky is the limit.

Two quick plugs:

• My company Second Coffee is focused on building a bunch of AI-first products (much larger than CSS Genie which was just a fun side project). If you have a burning need that constantly frustrates you, let me know!
• Also, I am separately working on a new type of community chat app. (If you run a community and are dissatisfied with Slack/Discord, or haven't found a suitable platform yet, please let me know!)