How I Built My First AI-Powered News-to-Video System as a Junior Developer

Around 2016 to 2018 I worked at a small startup as a junior .NET backend developer. The company had ten employees at most. I was the second developer and worked closely with a senior developer. This is an account of what that project taught me, including an architecture mistake we had to go back and fix.

A note on scope: this was a team effort. I will say where I worked directly and where I am describing what the team built together. I will also leave out internals I do not remember well enough to describe honestly.

What the product did

A user gave the system their preferred news websites. The system retrieved articles from those sites, summarised them, found relevant images, and produced a narrated video with audio, transitions, captions and the other production elements that make a sequence of images feel like a short news item instead of a slideshow.

Each stage fed the next, so a weak stage showed up in the final video. A weak summary weakened the whole video. Badly matched images made the video look careless however well it was encoded.

The summarisation came from a machine learning model our team trained using Python and machine learning libraries. It identified related ideas across different parts of an article and connected them into a shorter summary, without generating entirely new sentences. I am not going to name the model architecture, the training data or an accuracy figure, because I would be guessing.

Working outside the .NET comfort zone

I was primarily a .NET developer. This project pushed me into Python on the machine learning side and into video processing, where my experience was limited.

We also built an image-retrieval tool that scraped image search engines instead of relying on an image-search API. That choice means owning the fragility. A scraper depends on page structure you do not control, so it needs ongoing attention in a way a documented API does not.

A video is not images plus audio

I learned FFmpeg while working on video creation and processing. From the outside it looks like a command-line tool for converting files. The difficulty is the model of video underneath it, and that model has a lot of moving parts.

• Encoding and decoding. Video on disk is compressed. Reading it means decoding and writing it means encoding, and the settings you pick affect quality, file size and processing time.
• Frames. A video built from still images is a sequence of frames at a fixed rate. A picture that stays on screen for four seconds is many frames, and anything that moves or fades during those seconds is a slightly different image in each one.
• Bitrate. It is a budget. Set it too low and the picture breaks up, set it too high and files become large and slow to deliver.
• Audio. Narration has its own timeline. Picture, captions and sound all have to stay aligned, and small timing errors become visible mistakes.

If you are picking up a tool like this for the first time, the approach that pays off is a tight loop. Render a very short clip, change one setting, render again and compare. Video problems are often visual or audible before they are logical, so you need to see and hear the result, and small test clips keep each experiment cheap enough to repeat many times.

Then there were the effects. Some transitions and effects only made sense once I understood the mathematics behind them. To show what that kind of mathematics looks like, here are two general illustrations. They are examples of the idea, not a list of the effects we implemented. A crossfade is a weighted blend of a frame from the outgoing scene and a frame from the incoming one, with the weight moving from 0 to 1 across the transition. A constant rate of change can look mechanical, which is where easing curves come in. A slow pan across an image is a similar idea applied to position: a crop window interpolated between two places over time.

This is why an automated video pipeline is more than combining images and audio. Something has to decide how long each image is shown, line captions up with narration, join scenes with transitions, keep sound and picture synchronised, and encode the result into a format real devices can play. Each piece is manageable alone. Chained together, a timing mistake early in the pipeline turns into a visible glitch at the very end.

The MVP that assumed one user

The first MVP worked, and that was part of the problem. Its architecture considered a single user. With one person generating one video, nothing in the system contradicted that assumption. As usage increased and several people needed videos at the same time, performance suffered.

I do not have profiler traces or load-test numbers to share, and I would rather not invent any. What I can explain is why the assumption is so easy to make and so costly to keep. Producing a video is substantial work that takes time. A design that treats it like a quick request, or that quietly assumes nothing else is running, has no answer to the question “what happens when several of these start together?” In general, those jobs end up competing for shared resources and everyone waits longer.

None of this was carelessness. An MVP exists to show that the idea can work, and this one did. The architecture question only becomes visible later, once the first answer is yes and more people start depending on the product.

An MVP can reasonably skip scale. It should not skip the question of what its main assumption is, so you know what you will have to revisit.

Rebuilding parts of it

We rethought and rebuilt parts of the architecture instead of starting the product over. Asynchronous operations and concurrency handling were among the things that helped. I do not want to oversell that, because async alone did not solve the problem. Asynchronous code lets a process wait without tying up a thread. It does not make expensive work cheaper, and it does not decide how much of that work should run at once.

My own takeaway is about the question to ask. “Does this work?” is the right question for an MVP, but it is not enough on its own. “What does this look like with many users at once?” exposes different problems and leads to different decisions. For another story about revisiting an early architecture, see how I rebuilt an Excel-like web app after the first architecture failed.

What I took from it

We researched, experimented, tested, made mistakes, learned from them, improved the system and shipped it to users. That sequence is what the project taught me about the job.

✓

Research is part of the work. Nobody on a small team has already solved every part of the problem. Learning an unfamiliar tool properly is a normal engineering task.

✓

Test the architecture against the second user. Ask what happens when two people use the system at once, then ten.

✓

Be exact about what a technique fixes. Async, caching and parallelism each address a specific cost. Naming the cost tells you which one to reach for.

✓

Expect to revisit decisions. Some problems cannot be predicted in an MVP. A design that can be changed is worth more than one that tries to be final.

For any system, whether it is a Python tool, an API or workflow automation, the habit worth keeping is to ask early who will use it, how many at once, and which assumption would hurt most if it turned out wrong. You can see examples of my work in my selected work, and my background on the experience page.