About locode

Getting Help in the Age of LLMs

Like most developers we're captivated by the amazing things large language models are capable of and the potential they have to transform the way we interact with and use technology. One of the areas they can be immediately beneficial with is in getting help in learning how to accomplish a task or solving a particular issue.

Previously we would need to seek out answers by scanning the Internet, reading through documentation and blogs to find out answers for ourselves. Forums and particularly Stack Overflow have been a great resource for developers in being able to get help from other developers who have faced similar issues. But the timeliness and quality of the responses can vary based on the popularity of the question and the expertise of the person answering. Answers may also not be 100% relevant to our specific situation, potentially requiring reading through multiple answers from multiple questions to get the help we want.

But now, with the advent of large language models, we can get help in a more natural way by simply asking a question in plain English and getting an immediate response that is tailored to our specific needs.

Person vs Question

locode.dev is our attempt at providing a useful platform for other developers in this new age by enlisting the help of large language models to provide immediate and relevant answers to your questions. But instead of just using a single LLM to provide answers, we're using multiple models to provide different perspectives on the same question that we'll use to analyze the strengths of different LLMs at answering different types of questions.

Initial Base Line

For our initial dataset we're starting with the top 100k questions from StackOverflow and running them through a number of quality Open LLMs that we've found to perform great for answering programming questions:

Our initial pass will be to see how well each of these models perform on the StackOverflow dataset which we'll keep track of and publish on our Leaderboard page which we're also comparing against the highest voted and accepted answers on StackOverflow to see how well they measure up against the best human answers.

Future Work

After establishing the initial base line we'll look towards evaluating different strategies and specialized models to see if we're able to improve the quality of answers that can be provided.

New Questions

For new questions asked we'll also include access to the best performing proprietary models to active users as they ask more questions, including:

Open Questions and Answers for all

All questions, answers and comments is publicly available for everyone to freely use under the same CC BY-SA 4.0 license used by StackOverflow.

Help us improve Answers

You can help us improve the quality of answers by providing any kind of feedback including asking new questions, up voting good answers and down voting bad ones, correcting answers with inaccuracies or leaving comments suggesting improvements or adding additional context and clarifications to answers. Our most active users who help curate and improve the quality of questions and answers will have the opportunity to become moderators where they'll have access to all our models.

We also welcome attempts to beat the large language models by providing your own answers to questions. We'll rank new answers and include votes they receive from the community to determine the best answers.

We'll use this feedback to update our leaderboard and improve the quality of answers provided.

Feedback ❤️

We're still in the very early stages of development and would love to hear your feedback on how we can improve locode.dev to become a better platform for answering technical questions. You can provide feedback in our GitHub Discussions: