Subscribe
Topic

DevOps

DevOps for LLM applications: testing, review and deployment pipelines. Posts compare LLM gateways like LiteLLM and Portkey, AI code review tools like CodeRabbit and Greptile, CI/CD for LLM apps, and cover a Copilot Autofix command injection bug.

Articles

1-7
  1. LLM Evaluation Frameworks in Production: Comparing DeepEval, Ragas, Promptfoo, and TruLens Architecture, Synthetic Data Generation, Judge Calibration, and CI/CD Automation

    Automated evaluation is the primary engineering bottleneck in deploying reliable LLM applications. While traditional software engineering relies on deterministic unit and integration tests with binary pass/fail conditions, generative AI systems produce non-deterministic, open-ended natural language outputs. Relying on manual human review or ad-hoc prompting fails to catch regressions across prompt modifications, model version updates, and retrieval pipeline adjustments. To establish rigorous qu

    AI
    8 min read
  2. Automated AI Code Review in Production: Architecture, AST Context Assembly, Linter Hybridization, and Multi-Stage Noise Reduction

    Naively piping unified git diffs into a large language model and posting the raw output to GitHub or Bitbucket is a reliable way to degrade engineering velocity. While frontier models demonstrate high zero-shot reasoning capabilities, unconstrained code review bots suffer from high false-positive rates, superficial formatting nitpicks, hallucinated API misuse, and context blindness. When an automated bot generates twenty low-value comments per pull request, developers suffer review fatigue and r

    AI
    8 min read
  3. CI/CD for LLM Applications: Automated Regression Testing, Golden Datasets, and Pull Request Quality Gates

    Deploying generative AI applications into production environments requires a fundamental shift in software quality assurance. Traditional continuous integration (CI) workflows rely on deterministic assertions: given a fixed input, a function must return an exact expected output. Large language models (LLMs) break this paradigm because their outputs vary across runs, token probabilities drift with prompt alterations, and natural language responses cannot be validated with binary equality checks.

    AI
    6 min read
  4. RAG Evaluation Frameworks in Production: Architecture, Metrics, and CI/CD Trade-Offs for Ragas, DeepEval, TruLens, and ARES

    Production Retrieval-Augmented Generation (RAG) systems fail silently. Unlike traditional software pipelines that throw explicit exceptions on invalid states, a broken RAG pipeline produces syntactically fluent, confident prose that conceals severe underlying defects. When a user receives an incorrect response, the failure can stem from multiple distinct failure points across the stack: the query embedding failed to retrieve relevant chunks, the reranker discarded the critical passage, the chunk

    LLM
    7 min read
  5. GitHub Copilot Autofix Introduced Command Injection in Snowflake CI Pipeline

    A security vulnerability introduced by GitHub Copilot Autofix left a public Snowflake repository vulnerable to unauthenticated remote command injection for five days before an autonomous security agent discovered and exploited it, according to a technical disclosure published by Wiz Research on August 17, 2026. The incident highlights emerging operational risks at the intersection of automated code generation and CI/CD security, demonstrating how automated remediation bots can silently strip ou

    AI
    2 min read
  6. Cursor Launches Origin Code Hosting with Bidirectional GitHub Sync and Agent Reviews

    Cursor has launched Origin, a proprietary code hosting service currently in early beta across all paid subscription tiers. The platform introduces native Git repository hosting directly inside the Cursor ecosystem, competing with GitHub and GitLab while embedding AI agents directly into version control and code review workflows. Origin allows developers to create and manage repositories, review pull requests, browse code, and automate continuous integration workflows without leaving the Cursor

    AI
    2 min read