The Aha Moment

Taking AI filmmaking from curiosity to a career

For anyone who is an aspiring AI Filmmaker that wants to take AI filmmaking from curiosity to a career, this post is for you.

For those that have been following me, most of you know that a year ago I announced that I was going to set out on a ridiculous and most likely doomed adventure - creating a feature length film using AI.

But I’ve done this type of thing before… and as a result of failing at one thing and discovering something else, I’ve ended up getting myself into some pretty cool things - including founding the first large scale NLP and Machine Learning powered search engine called Powerset, which is now called Microsoft Bing, I also got to see the first email cross the chasm on a mobile phone, I also got to work at the Stanford Research Center on distributed voice recognition - which we not all call Siri, Google Voice, and Alexa - yada yada 5 companies in total
— each an amazing journey.

Nothing ventured nothing gained right? Curiosity is my king.

In this case, my crazy idea was to attempt to create a feature length movie called Shepard’s Tone - loosely inspired by Game of Thrones, Lord of the Rings , and a little bit from a few other movies as well — using AI. Just so you know I’m not crazy, I know my writing, directorial, and acting skill pale in comparison to the best of Hollywood. I had no illusions that I would win an Oscar for Shepard’s Tone - my goal was otherwise

— that in “the trying” I would discover something.

TLDR - I did. I’m creating a new company called ShotKit and you can sign up for early access below. But I urge you to read the Ah Ha moment story below.

So today, my regular posting returns… and I’m going to share the Ah Ha moment where I came up with an idea that came out of “the trying” of creating Shepard’s Tone.

First, some high level things.

1. There are a LOT of people who aspire to be AI Filmmakers

2. There is no “Film School” for AI Filmmaking

3. It’s nice to see an elephant dance, but it doesn’t mean that I’d take it to the ballet.

Today, AI filmmaking is just a curiosity, a fun pile of randomness
- but how do we take it from here and turn curiosity into a career.

Moreover, the problems that rise in an effort to create a feature length film are unique and massive. To shed some light on just how massive, and to share some hard lessons learned, here’s a summary of what you’d have to do in pre-production to prepare for a feature length film that used AI.

1. Films have scripts, scripts have sequences, sequences have scenes, and scenes are made up of the atomic unit of a SHOT. The fundamental building blocks of SHOT are

   1. Ingredients - PROMPTS and REFERENCE IMAGES

   2. Recipes - WORKFLOWS that utilize various AI services

2. Creating the ingredients for a film is a MAJOR pain in the butt

   1. For a shot - you need perfectly color-graded and scene matched images for characters, style, mood, and scene. The average shot requires 10 ingredients.

   2. For a film - there are on average 3,000 shots, so you need a library of a minimum of 30,000 ingredients.
      
      … and if you want enough reference images (ingredients) to cover any possible shot angel, distance, framing, f-stop, etc you’d need anywhere between 15 billion and 340 quadrillion - depending on how you calculated things.
      
      — check my math, but I think that’s a lot.
      

3. Creating the recipes for a film is a MAJOR pain in the butt

   1. For a shot - Each type of shot may require a different recipe - e.g. a workflow with different ingredients, different AI Tools, and different approaches.

   2. For a film - you need a vast library of known recipes that work.

Thus, if a film has 3,000 shots, you will need at least 30,000 ingredient reference images and will need upwards of 100 recipe workflows that don’t just use one monolithic AI generator, but probably use a bunch as each is good for solving different workflows.

And that’s just the visual side of things. There’s also the audio side.

1. Films have music, underscore, and mood which must perfectly fit in the scene.

2. Films have foley and environmental sounds that bring viewers in to the scene.

3. Films have sound effects which must perfectly match the action in the scene.

4. Films have the dialogue of the characters, which must be perfectly lip synced to the characters, match the characters look, feel, and actions, and must be mixed and mastered so that they sit in the scene properly.

… and that’s a major problem. Because I think a lot of people want to be independent filmmakers with a small, or even no, team.

So I had an idea… and this idea came from two gentleman who created something called Albion One and end up changing the entire world of film score composition
- Christian Henson and Paul Thompson.

They created a company called Spitfire Audio with the goal of making it easier for film composers to compose professional quality scores for Hollywood Films. And moreover, they realized that aspiring film composers faced a similar problem of ingredients and recipes.

They came up with an idea to create a sample libraries series they called Albion - and the first and most fundamental of that series was called Albion One. And what Albion One was, was an all-in-one composer kit for film composing - every (ingredient) instrument in the orchestra, every boom, whoosh, and hit of sound FX, and a vast library of mangled and warped drones, rhythms, and sounds to help create mood and tone all included of “how to use” recipes.

So with one single music kit - that cost $449 - you could create a film score.

— and I was a customer…

You see each year I challenge myself to learn something new… and several years ago I challenged myself to learn film composition… and boy was it difficult.

— that is until I found Spitfire and Albion One.

Before Albion one, I was lost in a sea of choices, complexity, frustration, and the pile of randomness monster was my nemesis. I’d have one instrument that came from Abbey Road, another from Teledex, and so on- none of the instruments fit together, none felt like they were in the same room. But after I bought Albion - where all of the instruments “fit together” - it was just the way to go
- a utility for me.

This first kit Albion One - covered the basic Hans Zimmer style of music composition, but then, one at a time, Spitfire release an Albion kit for every major music genre that occurs in film - thus Albion Tundra, Albion Neo, and so on, such that a film composer could by the all-in-one kit for any film project.

— and forever more… film composition changed.

So I had an idea… What if I created an Albion One for AI Filmmakers?

ShotKit - The Albion One for AI filmmakers - An all-in-one kit that “fits-together” for a specific genre of film, such that an AI filmmaker could use this kit as the basic building blocks to create a film.

For example, imagine there was a ShotKit for a cinematic world inspired by the genre of Fantasy ( e.g. Game of Thrones, Lord of the Rings, Skyrim). And that ShotKit included the ingredients and recipes necessary for composing any story within that world.

What’s in a ShotKit? Think of it as an AI specially trained as a plugin to the big boy AI generators.

1. Ingredients - a cinematic world represented in a VAST library of trained assets

   1. Pre-composed images and videos (e.g. establishing scenes, cut shots, etc)

   2. Reference prompt library - to go with the reference images and pre-composed images

   3. Reference image library - a vast library of color-graded, scene-matched reference images and videos for locations, points of interest, sets, objects, and characters that cover every shot distance, shot angle, shot framing, f-Stop, etc.
      
      Note: this is the BIG one - without a vast library of superb quality reference images - both SREF and CREF pre created - everything is much more difficult if not impossible.
      
      This is the heart of good input gives you good output. This is the heart of slaying the pile of random monster and getting great results.
      

   4. Post production library - color-graded, and depth mapped effects overlays, LUTs, and visual effects mapped to the cinematic world

   5. Dialogue - look and feel matched custom AI voices for each character

   6. Music and underscore library - mixed and mastered stems

   7. Foley and environmental library - mixed and mastered world sounds

   8. SoundFX library - mixed and mastered sound-matched effects
      

2. Recipes - A library of workflow masterclasses that teaches how to execute specific shot types with the library of assets.

   1. Shot Distances: Extreme Wide Shot, Wide Shot, Medium Wide Shot, Medium

      Shot, Medium Close-Up, Close-Up, Extreme Close-Up

   2. Shot Angles: Eye-Level Shot, High-Angle Shot, Low-Angle Shot, Overhead

      Shot, Dutch Angle, Over-the-Shoulder Shot, Point of View Shot, Extreme High Angle, Extreme Low Angle

   3. Framing Choices: Single Shot, Two-Shot, Three-Shot, Over-the-Shoulder, Over-the-Hip Shot, Close-Up Insert, Split Diopter Shot

   4. Camera Movements: Static Shot, Tracking Shot, Handheld Shot, Pan, Whip Pan, Tilt, Crane Shot, Steadicam Shot, Zoom, Rack Focus, Dolly Zoom, Roll, Arc Shot, Boom Shot, Push-In, Pull-Out, Overhead Tracking

… so I did a little experiment

In founder land - it’s important to do two things well in the beginning

1. Create a valid test of before and after to test the value of your idea.

2. Build a unicycle first, then bicycle… and so on prior to building a space ship.

So I set out to do two things

1. To test my idea, I took a scene that I had already created without a ShotKit, and then remade that scene using a mini ShotKit that I created.

2. I limited the size of my ShotKit to only include this scene… I’ll scale later.

IMPORTANT TO NOTE: This scene I chose took me 2 weeks without the ShotKit. Additionally, and perhaps more importantly, even when I finally achieve a good result
- I couldn’t reproduce it - instead, my nemesis showed up - the “Pile of Random” monster.

… and here the questions I wanted to ask myself

1. Did I save a massive amount of time while holding quality constant

2. Could I consistently reproduce the result 100% of the time

3. Was it fun? was it empowering? was it Mwah ha ha!

4. Did I see this as a “required” utility?

5. Would I pay for it?

—

The result: The video below is what I created able to create 10 minutes with my mini ShotKit and is a cut from the tutorial I’m releasing on my yet to be published Youtube channel SNUK3M ( aka Steve Newcomb - me) - I plan on giving everyone this ShotKit for free to get feedback. ( note: scared boy cut scene, SNUK3M stinger, and my AI song I did in Suno sold separately )

So now, after working with my own mini-ShotKit for Shepard’s Tone I can say the following:

1. 2 weeks to 10 minutes. Boom! - I literally pooped a Twinkie.

2. When I use the ShotKit and generate 100 results - 100 results are perfect. I killed the pile of random monster.

3. It’s fun and Mwah-ha-ha inspiring. Moreover It was instant gratification - I get to see my imagination turn into a scene in minutes - Nuts!

4. It’s pure utility for me. I would never do it any other way. It’s like after the first time I used Uber.

5. It seems so obvious the minute you do it once. I’d pay for this all day and twice on Sundays.

So in short…I found my Albion One!

Without a ShotKit, it takes me 2 weeks to create something that I can’t re-create reliably.
With a ShotKit - I get a repeatable result that’s actually higher quality in 10 minutes.

So what would people use a ShotKit for?

1. Stepping up their skillset and preparing for a career

2. Creating a portfolio of shorts showcasing their capabilities

3. Creating pitches so they can shoot a “filmed” movie

4. Creating a feature length AI movie

5. Creating pre-visualizations for existing films and shows
   

6. … and of course posting stupid social media posts :(

… and the list goes on.

There’s a ShotKit for that

Imagine there was a place, ShotKit.ai (… and yes I’ve already purchased it) - where you could go and buy your all-in-one ShotKits and workflow Masterclasses or every genre of filmmaking.

I couldn’t help myself, and this is a list of all of the “inspired by” ShotKits I want so…

• Emberfall (Fantasy) – A medieval kingdom with crumbling castles, enchanted forests, and war-torn battlefields.

• Neon Exodus (Cyberpunk) – A neon-drenched dystopian city with rain-soaked streets, towering holograms, and underground hacker hideouts.

• Ghostlight Manor (Horror) – A haunted Victorian mansion filled with flickering candlelight, creeping shadows, and spectral apparitions.

• Noir Eternal (Film Noir) – A moody 1940s detective world of smoky alleys, silhouetted figures, and rain-slicked cityscapes.

• Titan Outpost (Sci-Fi) – A distant space colony with barren alien landscapes, high-tech research labs, and ominous derelict spacecraft.

• Blood & Dust (Western) – A sun-bleached frontier town with saloons, dusty trails, and outlaw hideouts in the rocky badlands.

• Overdrive 86 (Action/Thriller) – A high-energy urban setting with car chases, neon-lit nightclubs, and tactical SWAT standoffs.

• Empire's Fall (Historical Epic) – A grand Roman empire setting with marble palaces, battle-worn legions, and secret political plots.

• AfterLight (Post-Apocalyptic) – A ruined wasteland with abandoned cities, scavenger camps, and eerie radioactive fog.

• Dreamwave (Surrealist/Experimental) – A bizarre dreamscape with floating islands, shifting physics, and mind-bending optical illusions.

But, and perhaps more importantly… this could a boon for studios and gaming companies.

• Internal/private ShotKits for their existing films to reduce production costs (e.g. b-roll, pre-viz, cinematics etc)

• Internal/private ShotKits created supplied to teams for internal pitches of new projects (e.g next stage development after first pitch)

• Training up their staff (e.g. storyboarding, shot listing, pre-viz, marketing)

• Trailer ShotKits to release to fans in concert with the release of their films and games - e.g. free marketing and buzz

• ShotKit fan fiction series for their existing IP - imagine a world in which AI filmmakers could tell their own stories with the real images, characters, sounds, and voices from iconic movies.

• Optioning “inspired by” ShotKits (e.g. EmberFall) - for super cheap low risk IP development.

But do I have Founder Fit?

Product Market Fit is often talked about in founder land - it’s when you build a product that has a fit in a market - preferably a large, healthy, and growing one.

But it’s Founder Fit that is often ignored, and for me just as important. By Founder Fit - what I mean is does it fit me. And in this case I think it does.

1. When I think about knocking down the gates that hold back young aspiring filmmaker I get totally giddy. And I mean totally giddy.

2. Don’t get me wrong, when I was younger… I was all about conquering the world and PRIDE PRIDE PRIDE…

3. But now I’m older, wiser, and because I’ve checked off a lot of the boxes on my bucket list, I realized I have a new bucket now - the one where I pay it forward - and help others to achieve their dreams.

I would measure the success of this endeavor not by how high I rise, but rather by how many people I help rise - studios and filmmakers alike.

… and now it’s time to scale that effort and take my unicycle, create my bicycle, and eventually get to my spaceship.

In an upcoming post, I’ll be releasing a 2 minute+ trailer video.

Sign up for early access below and give me your notes..
And for those that sign up at this vanguard stage, I’ll have a very special gift for you.

If you want to follow along subscribe to my The Brief (aka this substack) - I’ll be posting my behind the scenes updates here.

see you soon…