At this year’s DevDay, OpenAI focused on empowering developers with key updates to make AI more accessible, affordable, and customizable. The event introduced four major features: Vision Fine-Tuning, Model Distillation, Prompt Caching, and the new Realtime API. These innovations demonstrate OpenAI’s commitment to building a sustainable ecosystem that bridges the gap between cutting-edge AI capabilities and practical, cost-effective applications.
Vision Fine-Tuning: A New Frontier in Visual AI
One of the standout announcements from OpenAI’s DevDay was the introduction of Vision Fine-Tuning. This new feature allows developers to fine-tune GPT-4 models using images alongside text, vastly expanding the model’s ability to understand and process visual data. Industries such as autonomous vehicles, medical imaging, and visual search are expected to benefit from this feature, which promises more accurate and efficient AI-driven applications. This tool opens the door to creating customized AI models that excel at visual recognition, search functionality, and object detection, making it a vital addition for businesses reliant on visual data.
Vision Fine-Tuning is available now. OpenAI is offering 1 million free training tokens daily throughout October 2024. From November, fine-tuning with images will cost $25 per million tokens.
🔎 Get started by reviewing the documentation.
Model Distillation: Bringing Advanced AI to Smaller Models
OpenAI introduced Model Distillation, a feature designed to make AI more accessible by enabling smaller models to perform at levels comparable to advanced systems. This integrated workflow allows developers to use the outputs from models like GPT-4 to train smaller, more efficient versions, such as GPT-4 mini. Model distillation can significantly lower business computational costs while maintaining strong performance. For industries like healthcare or education, this means the ability to deploy AI solutions in environments with limited resources, such as community health centres or schools, without sacrificing quality.
Model Distillation is available now. OpenAI is offering 2 million free training tokens per day on GPT-4 mini and 1 million free training tokens per day on GPT-4 until October 31, 2024.
🔎 Get started by reviewing the documentation.
Prompt Caching: Reducing Costs for Repeated API Calls
OpenAI’s Prompt Caching is a welcome update for developers seeking cost savings. Many AI applications require repeated use of certain instructions or prompts, often increasing the cost of API calls. Prompt caching addresses this by automatically saving frequently used prompts. This feature is particularly useful for developers working with applications that involve repetitive tasks or consistent formats, such as customer support chatbots or content generation tools.
Prompt Caching is available now. Developers can use this feature with a 50% discount on reused input tokens for cached prompts.
🔎 Get started by reviewing the documentation.
Realtime API: Elevating Speech-to-Speech Applications
The new Realtime API opens up exciting possibilities for developers working on conversational AI and multimodal experiences. This API allows for real-time speech-to-speech interactions, eliminating the need for separate transcription, processing, and speech synthesis models.
The Realtime API ensures lower latency and more natural interactions, making it perfect for voice-activated applications in industries like customer service, healthcare, and education. With its ability to process audio instantly and support function calling, the Realtime API positions itself as a powerful tool for developers building voice-based AI systems, offering a competitive edge in conversational AI.
The Realtime API is currently available in public beta. It supports real-time speech processing and costs $0.06 per minute of audio input and $0.24 per minute of audio output.
🔎 Get started by reviewing the documentation.
With these major updates, OpenAI is streamlining the development of AI-powered applications while making it easier for businesses to scale and innovate. To stay updated on the latest AI news and trends, be sure to check out our weekly AI This Week series, where we cover everything from new developments to cutting-edge research.
