Hi there, Artem from Writingmate.ai here. This article delves into a quick comparison of four advanced AI models using ChatLabs: freshly released Gemini 1.5 Pro, and also GPT-4 Turbo, Mixtral, and DBRX Instruct. We compared their ability to handle coding assignment and vision-related tasks, aiming to provide a clear picture of their performance metrics.

About The Models

GPT-4 Turbo is the latest version of OpenAI's GPT-4 language model, with improved instruction following, JSON mode, reproducible outputs, and parallel function calling capabilities. It was trained on data up to December 2023.

Gemini 1.5 Pro is Google's latest multimodal AI model that can handle a variety of tasks like text generation, image classification, and object detection.

Mixtral 8x22B is a large-scale language model from Mistral AI, consisting of 8 experts with 22 billion parameters each. It uses a mixture-of-experts (MoE) architecture.

DBRX 132B Instruct is a new open-source large language model developed by Databricks. It outperforms existing open-source LLMs like LLaMA 2 70B on industry benchmarks for language, programming, math, and logic tasks. It uses a fine-grained MoE architecture with 36 billion active parameters.

Code Writing: Mixtral vs DRBX Instruct

Using ChatLabs AI, we asked both Mixtral 8x22B and Databricks DRBX Instruct models to write reverse proxy in python using markdown for formatting. Here's what we've got:

Mixtral 8x22B Result

DBRX Instruct delivered a dependency-free solution using Python's built-in http.server and socketserver modules for GET and POST requests.

1.7 sec to first token
29.8 tokens/sec
212 tokens
7.1 sec

DRBX Instruct Result

Mixtral, meanwhile, used the third-party Flask framework.

1.6 sec to first token
80.9 tokens/sec
558 tokens
6.9 sec

Conclusion

Further testing showed that both models sometimes suggested dependency-free solutions and other times used third-party libraries. Code quality and completeness were comparable overall.

Both models generated functional reverse proxy code quickly, with DBRX Instruct producing more tokens per second. But given the variability across multiple tests, we consider their coding performance fairly even.

AI Vision: GPT-4 Turbo vs Gemini 1.5 Pro

Next, we asked ChatLabs to compare GPT-4 Turbo and Gemini 1.5 Pro in Image Interpretation.

GPT-4 Turbo result:

The image humorously compares a large chicken-fried steak with bread buns placed on top to computer hardware components. The steak represents the main part of the hardware (labeled "GPU"), and the smaller bread buns represent the VRAM, indicating their relative size and importance in a playful way. The term "NVIDIA BURGER" at the top adds to the humor by linking the food items to NVIDIA, a well-known manufacturer of GPUs, suggesting that this meal is as powerful and substantial as a high-performance graphics card. This image is a creative and amusing way to visualize the relationship between a GPU and its VRAM using everyday items.

5.8 sec to first token
11.3 tokens/sec
130 tokens
11.5 sec

Gemini 1.5 Pro result:

The image is a meme comparing the size of a GPU to the size of its VRAM. The GPU is much larger than the VRAM, which is a common problem with many graphics cards. This can cause problems when running games or other applications that require a lot of VRAM.

⏱ 5.9 sec to first token
9.7 tokens/sec
57 tokens
5.9 sec

Conclusion

GPT-4 Turbo gave a detailed, nuanced interpretation, picking up on the humorous food analogy. However, it overextended the metaphor, suggesting the meal represented the power of a high-end graphics card - not quite the meme's main point.

Gemini 1.5 Pro, in contrast, provided a concise, accurate explanation: the meme compares the GPU's size to the much smaller VRAM, which can cause performance issues in games and apps that need a lot of memory.

While less detailed, Gemini 1.5 Pro's straightforward, literal interpretation captured the meme's essential meaning more effectively than GPT-4 Turbo's imaginative but less precise analysis.

What Is ChatLabs?

ChatLabs by Writingmate is a new platform that provides access to 20+ different AI models, including advanced models like GPT-4, Mistral Large, LLaMA, Gemini Pro, and Claude 3 Opus. With ChatLabs, you can use pre-built assistants to enhance productivity, creativity, and collaboration for both personal and team use. The key features of ChatLabs include:

• Choose between different AI engines: Access a wide range of leading AI models in one place to find the best fit for your needs.

• AI Assistants: Select unique AI characters, each trained to assist with specific tasks.

• Prompt Library: Make a list of proven prompts that you can use with any model to get the best performance.

• Talk with Your Data: Upload your company files, summarize them, and ask questions. The assistant will handle everything.

• Add Your Plugins: If you want more options, you can make plugins to connect ChatLabs with your CRM, Help Desk, or other systems.

• ChatLabs offers more flexibility and fewer restrictions compared to ChatGPT, allowing users to communicate with different LLMs.