Facebook has launched two new developments for AI benchmarking: Dynaboard and Dynascore that will help the researchers in creating machine learning models including the interaction of humans and the AI models.
The social media giant has been successful in bringing the two innovations into possibility following Dynabench, an AI tool that the California company made as an open-source platform for dynamic data collection in 2020.
Facebook Launches AI Benchmarking Models For the Researchers
On Tuesday, May 24, Facebook upgraded its Dynabench through its Dynaboard which is currently at the peak of machine learning process usage. According to VentureBeat, the social media firm said that Dynaboard creates a more convenient and similar model comparison without encountering test code bugs, data inconsistencies, and other issues.
For us who are usually on Facebook, these developments are not tackled normally. Since these are mainly used for AI projects, specifically in benchmark testing, the company has been focused to craft a new AI technology. These will process the algorithms faster and better so that the software will be guaranteed to always be in its latest version.
Mainly, the research teams are the targets of these improvements since they are the ones who will determine the progress of an AI project. Moreover, the companies have been dealing with benchmarking to seek the most appropriate machine learning model for the activity that they are conducting.
We're introducing Dynaboard, an evaluation-as-a-service platform for conducting comprehensive evaluations of NLP models. https://t.co/zhLk5zv70J. It enables dynamic, apples-to-apples comparisons dynamically, overcoming some of the key challenges in AI evaluation today. pic.twitter.com/dN4CCgTVG6
— AI at Meta (@AIatMeta) May 24, 2021
In connection to AI benchmarking, the introduction of Dynascore and Dynaboard has become a great challenge for computer scientists. While it's true that an AI model could outsmart its other networks in one trial, it could collapse in the next test.
The complicated nature of an AI-operated software could generate a myriad of results which will be an obstacle for an AI when it comes to algorithm comparison.
Dynaboard and Dynascore Rely on 'Fairness' Metrics
According to SiliconAngle, Facebook's Dynaboard could conduct an AI test through a cloud environment. Meanwhile, the Dynascore shares the same concepts from the microeconomics where various AI models are encased.
To test the Dynascore ranking, the accuracy of the trial can be made based on many factors such as its reactions to spelling errors.
"An NLP model should be able to capture that a 'baaaad restuarant' is not a good restaurant, for instance, to be considered [flexible] under challenging situations," the researchers from Facebook said. Moreover, they added they are also assessing the "robustness" of a model's prediction done through the addition of perturbations to the given. They will be helpful for the benchmark tests of the AI.
The software utilizes an unusual metric evaluation in the form of "Fairness." The Facebook researchers explained that the algorithmic bias and "fairness" challenges are now in the front seat of the AI community. The factors will first be evaluated and after that, the Dynascore ranking will be utilized by the Dynaboard so the researchers could make a relevant comparison between different AI models.
The experts conduct many tests which are done in multiple rounds. At each round, the difficulty of the tasks keeps on increasing. What transformed into challenges are the failed tasks that the model has done.
Somehow, the future of AI modeling could be seen as a prospective ground for potential experiments. Dynaboard and Dynascore are both promising to all people who are interested in machine-learning operations.
Related Article: Facebook Paid Events Tool: Pay-Per-View Feature to Launch for Live Sports Broadcasts
This article is owned by Tech Times
Written by Joseph Henry
