The “Shepard’s Tone” Project
The World’s First Open Sourced Episodic Series that uses AI from Script to Screen
I often say, "I’d rather throw up on stage than be caught sitting in the audience while history is being made." AI is scary, but I'm here not just to witness, but to shape, innovate, and challenge the paradigms of our time.
Throughout my entrepreneurial journey, I've consistently been drawn to areas of innovation, engaging deeply with transformative technological developments. I’ve been fortunate enough to be part of the founding teams that contributed to the development of voice recognition, the first email the crossed the chasm of the mobile phone, and perhaps most prominently the founding of the Powerset search engine which became Microsoft Bing. But beyond theses technological milestones, what truly fascinated me what the broader societal implications of these innovations. As my focus now transitions to the intersection of AI and filmmaking, I recognize both the opportunities and challenges that lie ahead. My objective in this space is to navigate the technological advancements while considering the broader ethical and societal aspects of integrating AI into the arts.
With that in mind, it’s time to throw up.
I'm excited to share the announcement of the "Shephard’s Tone” Project, my venture into open development and open sourcing of a live-action episodic series that seamlessly integrates AI from script to screen.
This project’s mission is multifaceted: Firstly, I aim to pull back the curtain on advanced AI techniques throughout each step of film creation. Secondly, I'm committed to openly addressing, head-on, the ethical, artistic, and legal challenges that arise in this integration. And thirdly, my goal is to complete this ambitious series, reflecting upon and maintaining the integrity of my artistic vision throughout. While I’m leading the way, I won't be alone on this journey; I've surrounded myself with a team of world-class artists who provide both expertise and creative insight —and perhaps most importantly… they yell at me and make sure I don’t ruin earth.
I have no idea if this is going to be a huge success, or a huge train wreck, but I do know that it’s going to be an interesting ride — I’d love to invite you all for the journey with me.
Now, let's delve into the approach and the scope of this project.
During the creation of "Shepard’s Tone", I'll be putting the SMURF stack to the forefront, embodying the nexus of innovation and accessibility in modern filmmaking. Each component of this avant-garde framework, from screenwriting to the final edit, integrates a specific form of AI—be it integrated AI, which seamlessly merges with core functionalities; orthogonal AI, offering distinct, complementary capabilities; or analytical AI, harnessing data to refine processes. To earn a spot in the SMURF stack, a software not only has to be AI-driven but also free or nearly free, user-friendly, operational on a typical home PC, and, optimally, open-source. By harnessing the full potential of the SMURF stack, I aim to not only elevate the caliber of "Shepard’s Tone" but also showcase the transformative power of accessible and intelligent tools in reshaping the cinematic landscape.
SMURF STACK DEFINED. The SMURF stack represents a cutting-edge framework set to redefine the film and television industry by integrating technology and creative processes. Constituting the acronym "SMURF" are the following components: Screenplays (with AI-driven tools assisting in the crafting of intricate narratives and shot lists); Modeling and Environment Building (utilizing open-source platforms and asset libraries to create realistic models and immersive settings); Unreal Engine (pioneering the realm of virtual production to blur the lines between pre and post-production); Rendering (leveraging AI-enhanced tools for faster and more efficient visualization of scenes); and Final Cut (using advanced editing suites that incorporate AI to optimize the post-production narrative flow). Collectively, these elements are poised to democratize and revolutionize filmmaking. If you’d like to read about the SMURF stack in detail, feel free to read my “Got SMURF” issue from The Brief.
For Pre-production, I'll be employing AI writing tools such as ChatGPT, Bard, and others for the foundational scripting elements. Visualizations, including concept art, storyboards, and shot lists, will be enhanced using advanced versions of Midjourney, Dall-E, Stable Diffusion, and Adobe’s Firefly.
When we move to the Production phase, resources like Blender, Unreal Engine, Kitbash3D, Big Medium Small Kits, MetaHuman and more will be central in bringing our series to life visually.
And in Post-production, DaVinci Resolve will play a pivotal role in editing and color grading. The visual spectacle will be further enriched with MAKEBIGFILMS VFX suite. For the auditory dimension, Boom Library will aid with Foley and SoundFX, and music will emerge from Logic Pro, my extensive sample library, and state-of-the-art AI music software.
Each step in these phases will be organized into detailed sprints, spanning 1-2 weeks. Each sprint will have daily, albeit concise, updates that shed light on specific tasks I'm working on. This includes a reflection on the ethical quandaries associated with these tasks, as well as a dedicated tutorial on the AI techniques being employed. At the sprint's end, I will release the final product achieved during that period, and accompany it with a thorough discussion around the ethical, artistic, and legal challenges I faced and navigated.
Throughout the journey of the "Shepard’s Tone" project, I'm committed to fostering an atmosphere of transparency and knowledge sharing. As part of this endeavor, any visual clips produced will be made available for free viewing on YouTube, ensuring that audiences can closely follow and engage with the evolution of the series. In addition, I will be venturing into the forefront of innovation by testing alpha features of various AI programs, delving deep into their capabilities and implications. But it's not just about the tools; it's about the community and the collective wisdom. To that end, I'll be conversing with key figures in both the AI and film sectors, aiming to capture their insights on the myriad ethical, artistic, and legal challenges I encounter. And, for those who wish for a more direct interaction, if there's enough interest, I'm more than willing to host office hours to answer questions, share insights, and foster a deeper dialogue around the project's ongoing developments.
By embarking on the "Shephard’s Tone” Project, my aspiration is not just to create, but also to foster understanding, sharing both the marvels and the intricacies of melding AI with filmmaking.
If you’d like to participate in any way, or just yell at me, feel free to contact me - steve.e.newcomb@gmail.com
What is The Brief and Who should read it?
"The Brief" is a platform that delves into the convergence of AI and arts. Its threefold mission encompasses sharing advanced AI techniques, addressing ethical, artistic, and legal challenges, and narrating the journey of "Shepard’s Tone" - an open-sourced live-action TV series. The project employs the SMURF stack, a combination of AI tools designed for affordability, user-friendliness, and home PC suitability. As the series unfolds, anticipate free YouTube releases, tests of AI tools in their alpha stages, industry expert interviews, and potential office hours for in-depth discussions.
Who am I?
I'm Steve Newcomb, a seasoned (meaning I’m old) Silicon Valley founder known for founding Powerset, which evolved into Microsoft Bing post-acquisition. I was part of the team sending the first email on a mobile device and contributed to modern speech recognition at SRI. I co-founded the first company introducing a 3D physics engine in Javascript and have supported vast open source projects, including NodeJS and jQuery - the largest open source project in the world. I've been a senior fellow at UC Berkeley’s Engineering and Business Schools and recently aided Matter Labs in securing $440 million for Layer 2 internet protocols.
I also am a lifelong composer and have my studio built in Sonoma, Ca.
If you are interested in contacting me, being interviewed, being helped, or yelling at me, my email is steve.e.newcomb@gmail.com.