yesterday at 10pm i was deep in conversation with a founder running an ai startup. eight hours of calls behind me. dozens of founders interviewed. everyone working on either neural nets, vision transformers, token optimization, rag systems, integrations for enterprise. but when i asked what proof of value means for developers in ai, the no one had a clear answer. they said and i quote “if you get an answer to that, please share.”
proof of value used to be simple.
you solved a business problem. shipped features. scaled systems. clear metrics. but ai scrambled everything. now founders look for weird signals. have you built an mcp server? touched agent architectures? worked with acps? they're checking if you're up to date, not if you're good.
the number of companies hiring
for ai in india is still evolving. most are early stage. the ones that exist care more about freshness than experience. four year old ai work? irrelevant. two year old work? ancient. this makes showing proof of value nearly impossible. but there's good news.
what all will we cover?
first, org structure
every technology shift reshapes
how companies organize. understanding
how ai companies are setup tells you
where opportunities live.
second, your fitment
frontend or backend, your path differs. we'll decode the core business problems companies actually face.
from there, learning
ai foundations and skills that matter.
for fe vs be vs devops engineers.
finally, the meat:
picking projects that demonstrate real value
1/ macro context (org structure)
whenever a new technology shift comes,
it always reflects in the way a team or the org is set up to deploy the technology. let's go way back to the industrial revolution to then, let's say, a tata was set up back in the day for a steel company. to the next revolution around computers. the way an ibm or an apple hardware division was set up. and then the internet came. to deploy internet based products fast we got fe, be and devops.
you cannot take an internet based company's structure and go and deploy like, you know, hardware. that setup won't work. and now with ai that’s changing.
ai org structures are nuanced.
i spent hours mapping job postings from openai, cursor, lovable, bolt, replit and more. this is my understanding after researching across hundreds of ai companies, looking at their job descriptions, whether it's foundational companies like openai, anthropic, deepmind. or it’s application layer ones like cursor, lovable, bolt, & replit. let’s dive in.
product engineering
frontend
you’ll build every surface a user touches. think chat, voice, and multimodal token streams. you’ll craft prompt editors, playgrounds, and in browser eval toggles, all held to a sub 300 ms p95 latency budget. the role is billed as senior frontend engineer or full stack core product engineer.
model services
backend
all code that talks to checkpoints. retrieval and vector db orchestration. agent runtimes, tool calling, control plane for model rollouts. developer tooling and experiment harness. an example listing is an ai applied engineer focused on reliability at lovable.
platform
infrastructure
autoscaling, spot instance bidding, cache layers. observability stacks for token cost and latency. (mostly seen in larger ai companies)
data & evaluation
new
you’ll write the code that speaks to checkpoints, manage retrieval layers and vector db orchestration, shape agent runtimes and tool calling, and run the control plane for model rollouts. you’ll also craft developer tooling and an experiment harness. an example job listing is an ai applied engineer focused on reliability at lovable.
reliability & safety
new
you’ll handle autoscaling, design cache layers, and build observability for token cost and latency. roles like this show up mainly at larger ai companies.
how does this change by
stage of the company?
early stage companies
at lovable, they hired a generalist product engineers when the team was still under fifteen. once headcount hit roughly twenty five, the first boundary appeared. product engineering split off from model services.
mid stage,
platform engineering emerges. primarily driven by the core need to implement a dedicated team that looks at models, evaluation and safety.
larger-ish companies
here the third split to safety happen. plus token optimization for cost, spend caps, guardrail filters. someone needs to own incidents when the model starts speaking latin.
if the company focuses on models instead of just the app layer, the org splits into training infra and inference infra. baseten already lists roles for model performance, site reliability, and forward deployed engineers. research is only now spinning up as its own track.
2/ where do you fit
this section gives you a high level view of what problems you should be solving, based on the role you're in. i.e. a front end dev or back end.
1/ frontend engineers
fe devs at ai companies are like ux therapists for unpredictable machines. your users expect chat interfaces that handle partial responses, voice inputs that feel natural, error states for errors that shouldn't exist. the model returns malformed json. hallucinates data structures.
the core jtbd (job to be done) is
to build trust interfaces. visualization for attention weights, confidence scores, reasoning chains. making black boxes feel less black without overwhelming users. evaluation interfaces everywhere. a/b testing model outputs. collecting human feedback. determining if the new model actually beats the old one.
my opinion on this: you still own experiences. but the medium of experiencing products is changing. voice is going to be the new frontend.
2/ backend engineers
honestly? it’s a lot about data.
your core job is to build platform systems that take care of all kinds of observability– evaluation, safety, accuracy & cost. the second part is all around building a platform for ai native integrations with external data pipelines. and the last part is about spee
monitoring becomes philosophy.
how do you alert on "the model got slightly dumber"? you build evaluation pipelines that run constantly. detect drift before users notice. implement guardrails for behaviors nobody predicted because nobody could predict them. and lastly is around latency (this is specific to certain businesses.)
3/ learning about ai
1/ start with foundations
even though everyone says they don't matter.
my friend & i used to work in an ai company about 10 years ago. he was the main ai person there. now at an OTT mnc working leading their ai.
i asked him before writing this article
how imp is foundation? "everything happens at the application layer now. he said" “foundations are not recommended tbh” then i asked how he debugs or solves a problem. i dug deeper, but how did you know where to find that issue? "oh, have a gut about how these models fail and why"
the goal is not deep expertise.
when someone mentions "catastrophic forgetting" or "gradient explosion," you need enough background to engage.
think of it as learning
a new neighborhood. you don't memorize every address, but you know the main streets :)
i reccomend
1/ watch karpathy's neural network series.
2/ deeplearning.ai’s foundational courses.
2/ what skills should you learn?
1/ python proficiency is core.
this has come out every single time. like new internet companies ran on javascript. ai runs on python man.
2/ some math classes will help
if you’re applying for companies that are ai first, i’ve reviewed their interview questions and they def ask questions around loss plateau, batch size & more. you need mathematical thinking. because model outputs are probability distributions pretending to be answers.
building the proof
let’s start with how we always pick any good project. we ask what are the top business problems right now? these will not be sexy projects, but def worth building.
data pipeline reliability
wastes countless hours. cleaning, deduplication, versioning, privacy compliance. build tools that handle messy real world data. show before and after metrics.
evaluation infrastructure
everyone’s flying blind without the right tools. most teams can’t even tell if their new model is better than the last. you’ll build frameworks that compare outputs, catch regressions, and surface subtle degradations. the goal is simple: make quality measurable.
latency reduction
model routing based on query complexity. sub 300ms responses for chat. instant feel for completions. progressive streaming that doesn't break. show p95 improvements.
two hiring managers told me that i am looking for freshness of the tech they’re using. don’t show me your projects from 5 years back on ai. while they are great, the world has changed.
i asked why does freshness matter so much? it’s simple. i want to know whoever i am hiring knows what’s latest. that can be an mcp server built or small models running on phones, browsers, embedded devices. webgpu implementations. anything that shows you know what’s happening in the ai space right now.
i asked, what would make you
say that if this person has done ____ i would hire them.
1/ open source contributions to ai projects
pick projects developers actually use. not documentation fixes or typo corrections. real features. look at issues labeled "help wanted" on major repos. start small but meaningful.
2/ side projects
but you have to connect projects to business metrics. increased model accuracy on production data by 12%. caught 89% of quality regressions before deployment. some kind of metric that the business cares about.
show adoption and impact.
1,000 developers use your library. your tool saves 10 hours per week for ml engineers. doing these 2 things on top of the projects you choose is a guaranteed hire.
what is low quality work?
paper reproductions
read as homework. everyone can follow a tutorial. unless you beat the original paper's numbers or add novel optimizations, skip it.
demo apps wrapping
openai's api scream beginner. chatgpt clones. basic rag demos. these flood every portfolio. they demonstrate api reading skills, not problem solving.
finding opportunities
if you're working, start internal.
that manual process everyone hates? automate it with ai. measure time saved. document adoption rates. real impact on real teammates beats theoretical projects.
between jobs?
build in public. pick problems you personally face. create tools other developers need. share progress on twitter, linkedin, github. let the community validate your direction.
follow the pain.
read discord channels for ai frameworks. what do people complain about? monitor github issues. what stays open longest? join ml twitter. what problems repeat? pain points are project ideas. or engage with communities. join eleuther ai, laion, hugging face discord servers. participate in discussions. share your experiments. get feedback before building. communities guide you toward real problems.
the founders i interviewed
weren't really looking for ai specialists. they wanted evidence of continuous adaptation. can this person unlearn and relearn every quarter? will they notice when the ground shifts again? do they solve real problems or chase trendy demos?
so i'll leave you with the question
that haunted me those eight hours of conversations. in a field where knowledge expires faster than milk, what does it mean to be senior anymore?