April 4  Vu Ha

In this edition we announce Harmonious.ai, an online paper reading and discussion forum. Harmonious was born at the AI2 Incubator as part of our efforts to keep track of advances in AI to advise founders. We decided to open (source) it to encourage wider participation from all AI builders. We also share some observations and learned lessons working alongside founders building pre-seed AI startups for the last seven years as we wrap up the first quarter of 2024. Topics include how to pick the right idea, assemble a strong founding team, harness improving AI capabilities, secure compute resources, and navigate a tough fundraising environment. ICYMI, last month we shared the news about AI2 Incubator’s $200M compute offering to startups.

A non-trivial percentage of my Twitter feed look something like this:

or this:

For AI builders (engineers, scientists, tinkerers), keeping up with AI advances seems daunting. There are too many Discord channels, X and Reddit threads, newsletters, and blog posts out there, covering everything from the latest LangChain feature to Elon Musk suing OpenAI. One useful tool we found is Shawn Wang (@swyx)’s AINews, which pulls AI news and discussions across various places and summarizes into digestible bites, using AI. AINews has been endorsed by Andrej Karpathy, Soumith Chintala, and others.

We believe there should be a place where AI builders can gain crucial insights into building real-world AI applications. This place should be focused on builders, created by builders, and maintain an extremely high signal-to-noise ratio. We started Harmonious.ai with this goal in mind. At this time, we will share weekly roundups of interesting AI research papers where we highlight and review in-depth one paper as the week’s spotlight paper. The weekly roundups also include a few noteworthy papers with shorter commentaries. Below are key points:

Audience

: AI builders. No sign-up is required. Anyone who signs up (with Google, GitHub, Twitter/X, or direct) can

contribute

: reviews, guides, learned lessons, commentaries, etc. Once signed up, you will automatically get a weekly digest (frequency can be adjusted).

Topics/Moderation

: Initially, we will focus on topics related to building real-world LLM-powered applications, including retrieval augmented generation. We are open to other topics such as tools, libraries, frameworks, techniques, guides, etc.

Off-topics

are ads (e.g. recruiting ads), AI drama (e.g. OpenAI’s board drama), and anything that is not directly relevant to building AI applications and technologies.

Moderation

is for now done by the AI2 Incubator.

Why start with paper review? Historically, the time between the publication of a seminal paper (e.g. Attention/Transformer in July 2017) and its broad impact (GPT-2 in February 2019, GPT-3 in July 2020, ChatGPT in December 2022) is measured in months/years. The ChatGPT moment changed everything. While we may not be able to predict the next Transformer-level breakthrough paper today (and start the next OpenAI promptly), we are seeing ideas/techniques in research papers implemented in matter of weeks or days (the LLM-Lingua2 paper is a noteworthy paper for the week of March 18; it is already integrated into both LlamaIndex and LangChain). We have seen tremendous interest from AI builders to follow and read the latest research in AI (see for example the meeting on How To Read AI Research Papers Effectively organized by Andrew Ng’s Deeplearning.ai). Papers give us a peek into top of mind challenges for researchers pushing the state of the art, independent of frameworks, libraries, implementations, and organizations.

For the weekly paper roundup, we currently use Hugging Face’s Daily Papers as a source, which has about 50 papers per week (shout out to AK (@_akhaliq) for the great work selecting these papers and sharing with us daily). Feel free to ask us to review papers you are interested in by posting in the Review Request category. Below are some initial spotlight papers:

Please check out Harmonious.ai, read the content, share it with your network, and contribute your own insight.

Our last Insight, titled Trough of Disillusionment, conveys a sober sense of the difficult challenges founders face in building AI products on the path to creating enduring companies. In the startup seed funding world, we are deep in the trough territory, a wild swing of the pendulum from just twelve months ago when OpenAI announced GPT-4 and plugins. Seed investors go from funding pre-product companies to passing on ones with 6-figure revenue. At the AI2 Incubator, we receive hundreds of applications every week and remain optimistic about the opportunities available for startup founders. To navigate the early days of founding a company (from idea to seed funding), we believe the key ingredients are 1) picking the right idea, 2) assembling a strong founding team with market and technology fit, 3) embrace both market and technical challenges, securing AI compute if necessary, and 4) be extremely prepared for a challenging fundraising journey.

The first challenge founders face is identifying a compelling problem to solve with AI. YC’s Jared Friedman’s advice on How to get startup ideas is an excellent context for this discussion. Jared suggests the following seven ways to pick the right idea: 1) pick a problem in your domain of expertise (e.g. SnapDocs, MixPanel), 2) build something you wish someone build for you (e.g. DoorDash), 3) follow your passion (e.g. Boom), 4) exploit emerging tech (e.g. iPad -> PlanGrid), 5) variation of a theme (e.g. Amazon Go -> Standard Cognition), 6) crowdsource your ideas, and 7) look for industries that seem broken (e.g. payday lending -> LendUp). The fourth point, emerging tech, is clearly applicable: LLMs, agents, and multi-modal generation all present new opportunities. We want to focus particularly on the first point, domain expertise, and the last one, broken industries, as they are related.

In September 2022, Sequoia Capital published a post discussing GenAI's potential to generate trillions of dollars in economic value across all industries by automating or enhancing knowledge and creative work. Since then, startups like Harvey for legal services and Devin for software engineering have emerged, aiming to validate this prediction. However, there are lesser-known industries ripe for GenAI innovation that have not yet drawn adequate attention from entrepreneurs and investors. These industries are broken in the sense that much of the work is currently performed manually by highly trained, well-paid professionals. By using AI to automate or aid these processes, we could achieve productivity gains comparable to those seen in the industrial revolution. In ten years, we may look back and wonder how things ever functioned without AI.

Ideally, founders should have years of experience living or observing these inefficient workflows. As we evaluate applications for the AI2 Incubator, we immediately focus on this aspect. A strong founder-market fit has always been important. Today, when competition is intense and funding is challenging to secure, we believe it is even more critical.

The final aspect to consider is the boldness of the vision from a technological standpoint. Often, the most significant customer pain points are also the most technically challenging problems to solve, not chat-your-PDF variety. By tackling difficult and ambitious problems that matter to customers, founders can distinguish themselves from their competitors and build strong differentiation. One way to assess the boldness of the vision is to look at the gap from a proof-of-concept to production readiness: the larger the gap, the more ambitious the vision. Unsurprisingly, Paul Graham discussed a related concept called schlep blindness. To overcome the significant technical challenge of going from prototype to production, one needs the ability to persevere and solve myriad schleps along the way.

Not all founders possess a unique insight into inefficient processes in an industry they've spent years working in. This is often true for founders who are engineers and scientists, having spent most of their careers at major technology companies. They've honed their skills in building scalable systems, intuitive user experiences, and large AI models, but may not have experienced inefficient processes in less well-known industries. We suggest that these founders partner with co-founders who bring that insight and a good founder-market fit. The AI2 Incubator has effectively applied this approach of pairing co-founders with complementary skills and experiences on numerous occasions.

The increasing availability of LLMs from various providers such as OpenAI, Azure, Google, Mistral, and Anthropic, is fantastic for founders exploring use cases in their domains. However, fully leveraging these capabilities, particularly in light of a company's ambitious technical vision, can be challenging. For the best chance of success, startups may need an experienced chief AI engineer or scientist to close the gap between prototype and production. This person doesn't necessarily need to be a co-founder, but should be part of the founding team as one of the initial key hires.

In the past, we consistently recommended early-stage startups to stick with OpenAI’s GPTs, despite the emergence of numerous competing models in the last 12 months. However, based on our recent observations, Claude 3 has now emerged as a superior option for many use cases. We expect OpenAI to strike back soon. The fierce competition in this space, funded by an abundance of capital, has resulted in rapid and exciting progress in AI, opening up new possibilities for founders who can aim higher in terms of what they can build to serve customers. We've also worked with startups requiring substantial AI compute to train their own large models. This need motivated our effort to secure the $200M AI compute. We believe this resource will be crucial to help these companies make deep technical progress while building toward product market fit.

Let’s say you follow our advice so far: 1) pick a broken workflow in an industry underserved by technology, 2) layout an ambitious vision to fix that problem, 3) putting together a solid founding team with strong founder-market fit and technology/AI know-how, and 4) secure AI compute resource if needed. What does it take to get seed funding and scale the company? Investors typically look at the combination of team, market, and traction. Assuming that the team is solid, let’s look at the questions around market and traction.

When it comes to assessing market opportunities for GenAI companies, we found that conventional wisdom may be challenged. Jared Friedman discussed the topic of entering markets with few competitors:

Reconciling this advice with Sequoia Capital's vision of GenAI's trillion-dollar value creation is challenging. Such a massive wave of innovation would likely redefine market boundaries and establish completely new markets across industries. Investors are well aware of this. However, in the current conservative climate, they are cautious about venturing into markets that are either unfamiliar or have traditionally been challenging for entrepreneurs prior to GenAI. We recommend focusing on traction to address this concern. Demonstrate to investors undeniable evidence of customer demand such as extensive usage, strong word-of-mouth interest, and enthusiastic VC reference calls. Achieving this level of traction before a seed raise may be challenging, particularly considering the required AI sophistication. Therefore, founders may consider securing a smaller amount of pre-seed funding. Unbiased opinion: we think the AI2 Incubator is a strong option to consider 🙂.

We welcome and strongly support bold visions applying AI across industries. We believe we are well-prepared to support our founders given our experience and track record in incubating AI companies, our community of highly technical builders, the Harmonious.ai forum, and the $200M AI compute we recently secured.