![](https://crypto4nerd.com/wp-content/uploads/2023/07/0MJDAfeaCSuYNJwU-1024x512.png)
With LangKit, you’ll be able to extract and monitor relevant signals from Hugging Face LLM models, such as:
Try the Hugging Face + LangKit example in Google Colab.
Monitoring LLM Performance with LangChain and LangKit
Large language models (LLMs) serve as core pillars in today’s AI powered applications. From powering customer support interfaces, generating content, enhancing predictive text, and streamlining information retrieval, these models have emerged as a fundamental tool for most organizations. Read more on WhyLabs.AI
Safeguarding and Monitoring Large Language Model (LLM) Applications
Large Language models (LLMs) have become increasingly powerful tools for generating text, but with great power comes the need for responsible usage. As LLMs are deployed in various applications, it becomes crucial to monitor their behavior and implement safeguards to prevent potential issues such as toxic prompts and responses or the presence of sensitive content. Read more on WhyLabs.AI
Monitoring Large Language Models in Production using OpenAI & WhyLabs
In this workshop Sage Elliott shows how to monitor Large Language Models (LLMs) in production using Hugging Face, WhyLabs, and the LangKit library.
- 🔎 Understand: Monitor changes in behavior to evaluate system prompts, responses, and user interactions
- 🛡️ Guardrail: Configure acceptable limits to indicate things like malicious prompts, toxic responses, hallucinations, and jailbreak attempts
- 🚨 Detect: Set up monitors and alerts to help prevent undesirable behavior
whylogs v1.2.5 has been released!
whylogs is the open standard for data logging & AI telemetry. This week’s update includes:
- Only pass necessary columns to row UDF
- UDF signature update & Python data types
- Ranking Example — documentation upgrades
- WhyLabs Writer — tag known custom performance metrics
See full whylogs release notes on Github.
LangKit release 0.0.7 has been released!
LangKit is an open-source text metrics toolkit for monitoring language models.
- Add encoding definition when loading themes json
- LLM behavior monitoring example
See full LangKit release notes on Github.
Join the thousands of machine learning engineers and data scientists already using WhyLabs to solve some of the most challenging ML monitoring cases!
Request a demo to learn how ML monitoring can benefit your company.
See you next time! — Sage Elliott, Technical Evangelist.