Building an AI Meeting Companion with AFM-4.5B and llama.cpp
Ever been in a meeting thinking, “I should be taking notes, but I’m too busy actually participating”? Or worse, walked away with no idea what your next steps are?
Indeed, commercial tools from Zoom, Microsoft, and others provide AI-powered companion features; however, they operate entirely in the cloud, which raises significant privacy concerns for many teams and organizations.
So I thought, ‘What if I could prototype these features locally, on my laptop? Would that be possible?’
In this video, using a real-life meeting that I attended, I simulate a live transcript and run the following features in real-time:
✅ Summarizing the meeting so far,
✅ Suggesting questions to ask to make the meeting more productive,
✅ Chatting with the ongoing transcript,
✅ Writing personalized follow-up emails.
And there’s room for more. All wrapped in a sleek Gradio app running local inference with the AFM-4.5-Preview foundation model.
No LLM in the cloud. No data leakage. Just a private, powerful AI assistant that could easily tap into internal knowledge sources. The sky’s the limit.