Skip to main content

discussion 2024-07-24

Summary​

In the Discord chat, users engaged in technical discussions comparing various AI models' performance on ARC-AGI tasks without neural networks. Terafo mentioned DeepSeek Coder, which released an updated checkpoint today, suggesting it might rival Sonnet 3.5 quality and is significantly cheaper (50x less expensive). Pvncher expressed anticipation for the new model's performance, emphasizing trust in models to deliver optimal answers and excitement about the competitive landscape heating up. Carter_¡•○●■ joined the server later but did not contribute further to the technical discussion.

FAQ​

  • What is the DeepSeek Coder project?

  • terafo: The DeepSeek Coder exists as a project with an ARC-AGI score greater than 25% without using neural networks, and they release updated checkpoints regularly.

  • How does the quality of DeepSeek Coder compare to Sonnet 3.5?

    • terafo: The DeepSeek Coder is almost at the same quality level as Sonnet 3.5 in coding tasks.

  • Is there a significant price difference between two models being compared, and how does it affect their performance?

    • pvncher & terafo: One model is 50 times cheaper than the other with only a minuscule difference in performance. This price difference allows for more opportunities to improve over Sonnet 3.5.

Who Helped Who​

  • terafo helped Yosef Frost with finding a GitHub project by suggesting DeepSeek Coder, which had updated ckpt released.
  • pvncher helped Carter_¡•○●■ with joining the conversation and getting up to speed on the topic being discussed about ARC-AGI models' performance and pricing differences.

Action Items​

Technical Tasks:

  • Compare DeepSeek Coder's performance with Sonnet 3.5 (mentioned by pvncher)
  • Evaluate the cost-effectiveness of a model that is 50x cheaper than another, considering minimal performance difference (discussed by terafo and agreed upon by pvncher)

Documentation Needs:

  • No specific documentation needs were mentioned.

Feature Requests:

  • No specific feature requests were made in the chat.

Community Tasks:

  • Engage with new server members like Carter_¡•○●■ (implied by pvncher and terafo's welcoming messages)