The contest is over - Thank you for your contributions
|
See Linked In Post
As preparation for competing in the challenge we ran these crowdcast sessions:
Summary
Increasingly software plays a role in every aspect of our lives, from managing Covid-19 and our finances, to interacting with other humans in education, work, and entertainment. There are critical shortages of software developers and experts across the world. With large numbers of services and apps in popular programming languages and over 200 billion lines of mission critical legacy COBOL code, having methods to analyze, develop, and maintain software in more automated ways is essential for progress. As a step along the way to automation, join the challenge to find programs that perform similar tasks. The team that makes the best prediction wins. The challenge is based on an extract from Project CodeNet which is a large-scale dataset with approximately 14 million code samples, each of which is an intended solution to one of 4000 coding problems.
The goal of this challenge is to introduce data scientists to AI for code concepts leading to improvements in the way software is produced and maintained.
Useful Links
Background on Call for Code
Call for Code is a global program that invites developers and problem-solvers around the world to build and contribute to sustainable, open source technology projects that address social and humanitarian issues, while ensuring the top solutions are deployed to make a demonstrable difference. Created by David Clark Cause and supported by Founding Partner IBM, Charitable Partner United Nations Human Rights Office, and Program Affiliate The Linux Foundation, Call for Code has grown beyond a single global challenge into an always-on, tech-for-good platform with multiple ways to get involved.
The Challenge Synopsis
Working in collaboration with Women in Data Science, we will:
- Provide education on the codenet dataset, code representation, and NLP (Natural Language Processing)
- Help you form or be part of a team
Working in collaboration with your team, you will:
- Build your models and submit your predictions
- Join crowdcast sessions to learn about NLP, and code representation, network with other participants, and mentors
We will announce the winning team in January 2022 ---(or soon after when all checks have take place!)----
You will find a lot more information on the leaderboard http://ibm.biz/cfcsc-codenet-lead
The Prize
One winner at the top of the final leaderboard at the end of January 2022 (or when IBM declares the contest closed) gets 5K US Dollars.
Registering for the Contest
How to Win
The winning team will ultimately depend on the outcome of the of the 'final phase' of the challenge. The ranking of a solution depends on two criteria: the accuracy of the predictions and the submission time of the solution. The accuracy is calculated as follows: number of correct predictions/ total number of predictions. The key metric used is 'meta-score', which accounts for both accuracy and submission time. A higher accuracy score will always result in a higher meta-score and therefore a higher position on the leaderboard. For submissions with identical accuracy scores, the earlier submission will always rank higher. Potential winners have to submit a description of their method and a URL to a github repo with working code to allow verification of the eligibility of the submission. The team with the highest meta-score (as long as verified as eligible) will be the winner."
You will find a lot more information on the leaderboard http://ibm.biz/cfcsc-codenet-lead
The Phases of the Contest
| Phase Name |
Start in UTC & US Eastern |
End in UTC & US Eastern |
Crowdcast |
Experimentation phase
|
2021-10-19 - 23:59:00 UTC
2021-10-19 - 19:59:00 US Eastern
|
2021-11-09 - 23:59:00 UTC
2021-11-09 - 18:59:00 US Eastern
|
October 27, 2021
Call for Code Spot Challenge for CodeNet - Setting up your system
https://www.crowdcast.io/e/call-for-spot-challenge
|
| Dev phase |
2021-11-09 - 23:59:00 UTC
2021-11-09 - 18:59:00 US Eastern
|
2022-01-26 - 23:59:00 UTC
2021-01-26 - 18:59:00 US Eastern
|
|
| Final phase |
2022-01-26 - 23:59:00 UTC
2022-01-26 - 18:59:00 US Eastern
|
2022-01-28 - 23:59:00 UTC
2022-01-28 - 18:59:00 Eastern
|
https://www.crowdcast.io/e/spot-challenge-for-4 |
The Terms and Conditions
Please do make sure you have agreed to the Participation Agreement for the Call for Code Spot Challenge for CodeNet before you start submitting to the leaderboard. (See Registering for the Contest Section)
- No IBMers or Red Hatters can participate to win the prize - although they can experiment and build models
- A contestant can have one account only on the leaderboard and can be in one team only after the first phase
- Teams can be up to 4 people, with at least half the team identifying as women or non-binary
- Teams must be registered on this leaderboard by January 27, 2022
- No team mergers are allowed
- IBM can restrict the number of teams competing
- No sharing of notebooks and models privately between teams is allowed unless you make the content available to all
- At the end of January 2022, or when IBM declares the contest closed, the leaderboard determines the winning team
- The top teams on the final leaderboard will be asked to share their models and scripts - and provide information on the tools they used as well as any other open datasets they incorporated
We hope that you will join us.
#GlobalAIandDataScience#GlobalDataScience