What's new?

Evals 101 - AI Agent Evaluation Explained - 4 Pillars Every Team Should Measure Your AI agent gave the right answer, but did it do the job right? This full series explains how to evaluate AI agents in production, from task success to tool calls, latency, cost and safety. We follow one customer-support refund agent through every chapter and score it on the four pillars of agent evaluation: outcome, trajectory, operational performance, and safety and governance. You'll see when to use code checks, when to use LLM-as-a-judge, and when you need human review. What you'll learn: What evals are and why agents need more than single-response tests Task completion rate, factual accuracy, groundedness and user-experience scoring Tool selection accuracy, argument precision, step efficiency and error recovery Latency, cost per task, and consistency (pass@k vs. pass^k) Policy adherence, guardrails, prompt-injection and red-team testing, content safety A checklist of questions to ask your team before shipping an agent Chapters: 00:00 Why Evals Matter 01:04 Four Pillars Overview 02:12 Who Scores Evals 03:08 Outcome Metrics 07:23 Trajectory Checks 11:52 Operational Performance 15:33 Safety and Governance 18:18 Wrap Up and Team Questions Good for AI engineers, product managers, QA teams and anyone shipping LLM agents. #AIAgents #LLMEvaluation #AgentEvals #LLMasaJudge #AISafety #MLOps #GenerativeAI My Linkedin: https://www.linkedin.com/in/jacinthp/ --------------------------------------------- Github: https://github.com/jacinthpaul --------------------------------------------- Website: https://www.programstrategyhq.com/ ---------------------------------------------

Preparing for a Claude Certification exam? Updated exam content goes live on December 7, 2026. If you're ready, book and take your exam before the deadline. Exams affected: • Claude Certified Associate – Foundations • Claude Certified Developer – Foundations • Claude Certified Architect – Professional Key dates: 📅 Nov 17, 2026: Updated exam guides released 📅 Dec 7, 2026: Updated exams go live Which guide should you use? ✅ Testing before Dec 7: use the current exam guide ✅ Testing on or after Dec 7: use the updated guide Don't wait. Schedule your exam today. #Shorts #Claude #Anthropic #ClaudeCertification #AICertification #ClaudeDeveloper #ClaudeArchitect #AIExam #CertificationExam #TechCareers

Is your CLAUDE.md helping Claude Code or slowing it down? Run these 5 checks, 20 points each: ✅ Under 200 lines ✅ Specific, checkable rules ✅ Only what Claude can't infer ✅ Hard rules in hooks, not prose ✅ Imports, not copy-paste What's your score? Full breakdown in the main video. #ClaudeCode #CLAUDEmd #AIcoding #DeveloperTips #Shorts

CLAUDE md Explained | Best Practices for Claude Code Teams How many times this week did you explain your project to your AI assistant? CLAUDE.md fixes that. It's one file Claude Code reads at the start of every session, so your commands, conventions and architecture carry over to every chat and every engineer on the team. In this video I cover what CLAUDE.md is, the three levels (user, project, directory), what to include and what to leave out, six best practices, how to split context into product.md and design.md with imports, and how AGENTS.md lets you share one instruction file across Claude Code, Codex, Cursor and GitHub Copilot. 0:00 - Intro 1:24 - Chapter 1: What is CLAUDE.md 3:36 - Chapter 2: The Levels 6:26 - Chapter 3: Best Practices 11:25 - Chapter 4: Beyond One File 13:07 - Chapter 5: AGENTS.md 16:20 - Outro Blog: https://www.programstrategyhq.com/post/claude-md-explained-best-practices-for-claude-code-teams Resources Written guide and scoring rubric: https://programstrategyhq.com https://code.claude.com/docs/en/memory https://agents.md Subscribe for more on AI-native engineering, spec-driven development and agentic delivery. #ClaudeCode #CLAUDEmd #AGENTSmd #AIcoding #Anthropic #DeveloperProductivity My Linkedin: https://www.linkedin.com/in/jacinthp/ --------------------------------------------- Github: https://github.com/jacinthpaul --------------------------------------------- Website: https://www.programstrategyhq.com/ ---------------------------------------------

Prompt Engineering, RAG & Fine Tuning Explained | AWS AI Practitioner | AIF C01 | Ep 4 When should you use prompt engineering, RAG or fine-tuning? Learn how to improve AI responses, choose the right approach and evaluate the results, with practical examples and no coding. In Episode 4 of AI Fundamentals, we explore “Making Models Work” through topics from AWS Certified AI Practitioner (AIF-C01) Domain 3: Applications of Foundation Models. If your AI assistant gives a confident but incorrect answer, should you rewrite the prompt, connect your documents or train the model further? This episode helps you understand which problem each approach solves. 📚 WHAT YOU’LL LEARN • The four elements of a clear prompt: instructions, context, input data and output format • Temperature, top-p, top-k, token limits and stop sequences • Zero-shot, one-shot and few-shot prompting • Chain of thought, self-consistency and tree of thoughts • Prompt templates, context engineering and Amazon Bedrock Prompt Management • Prompt injection, jailbreaking, data poisoning and sensitive information exposure • How Retrieval Augmented Generation (RAG) works, from chunking and embeddings to retrieval and answers • How Amazon Bedrock Knowledge Bases supports RAG applications • Fine-tuning, RLHF, continued pre-training and model distillation • AI evaluation using ROUGE, BLEU, BERTScore, perplexity and LLM-as-a-judge • Three exam-style practice questions with answer explanations 🎯 WHO THIS VIDEO IS FOR AI beginners, project and program managers, product managers, business professionals and anyone preparing for AWS AI Practitioner who wants to understand the decisions behind an AI application. ▶️ CONTINUE THE AI FUNDAMENTALS SERIES Watch the series 💬 YOUR TURN For your own AI use case, what would you improve first: the prompt, access to information or the model’s behaviour? Share your example in the comments. Subscribe to Program Strategy HQ for practical AI learning. Next: responsible AI, security and governance. Presented by Jacinth Paul Program Strategy HQ Visit Program Strategy HQ #AWSAIPractitioner #PromptEngineering #RAG My Linkedin: https://www.linkedin.com/in/jacinthp/ --------------------------------------------- Github: https://github.com/jacinthpaul --------------------------------------------- Website: https://www.programstrategyhq.com/ ---------------------------------------------

Last week, I said I was going to build something around Wealth Management as part of GrowthX Build Sprint S4. Well, here it is! 🚀 Introducing Kosh — a personal wealth management planning tool with Insurance as v1. 👉 Try it here: https://qualified-squirrel-308.convex.site/ It's been a great experience building this over the past week, with Codex doing a lot of the heavy lifting. From an idea to a working product in just a few days. Still plenty to improve, but that's part of the journey! I also put together a short product launch video (attached). A big thanks to UD for the excellent session on how to create a product launch video. Lots of practical takeaways! Now comes the interesting part — will anyone actually pay for it? 😄 Next week is all about GTM (Go-To-Market). Finding potential users, getting feedback, understanding what they really need, and seeing if I can find someone willing to pay for the service. Building something is one thing. Getting people to use it, and eventually pay for it, is a completely different challenge. Looking forward to learning that side of things. The code is also available on GitHub: https://github.com/jacinthpaul/build-sprint-experiments Would love for you to give Kosh a try! Play around with it and let me know what works, what doesn't, and what you'd like to see improved. My DMs are open. One week of building done. One week of finding users and iterating ahead! #GrowthX #BuildSprint #BuildInPublic #Codex #WealthManagement #AI