[ad_1]
Developing the ability to notice massive volumes of data while maintaining quality is a function of the life cycle of model development that companies often underestimate. He uses a lot of resources and needs specialized expertise.
At the heart of a successful machine learning / artificial intelligence (ML / AI) initiative is a commitment to high quality training data and a path to proven and well-defined quality data. Without this quality data pipeline, the initiative is doomed to fail.
Computer vision or data science teams often turn to external partners to develop their data training pipeline, and these collaborations drive model performance.
There is no single definition of quality: “quality data” is completely conditioned by a computer vision or machine learning project. However, there is a general process that all teams can follow when working with an external partner and this quality data path can be divided into four default phases.
Notice criteria and quality requirements
The quality of training data is an assessment of the adequacy of a data set to fulfill its purpose in the use of ML / AI in a particular case.
The computer vision team needs to establish an ambiguous set of rules that describe what quality means in the context of their project. Ad criteria are a collection of rules that determine which objects should be noticed, how they should be noticed, and quality objectives.
Accuracy or quality objectives define the lowest acceptable outcome for metric assessments, such as accuracy, recall, accuracy, F1 score, and so on. Typically, a computer vision team will have quality objectives as to how exactly the objects of interest were classified, how exactly the objects were located, and how the relationships between the objects were accurately identified.
Staff training and platform configurationn
Platform configuration. Task design and workflow configuration require time and specialization, and tools for making specific notes are required for each. At this stage, the data science team needs a specialized pair to determine the labeling tools, classification taxonomies, and annotation interfaces, and how to configure the best for performance.
Employee testing and scoring. To accurately label data, reviewers need a well-designed training curriculum to properly understand the reporting criteria and the context of the domain. Advertising platforms or external partners should ensure accuracy by actively monitoring the ability of observers on gold data tasks or when a highly qualified employee or administrator changes judgment.
Basic truth or gold data. Ground truth data are essential at this stage of the process as a basis for scoring staff and measuring output quality. Many computer vision teams are working on a ground-based truth data set.
Sources of authority and quality assurance
There is no single quality assurance (QA) approach that will meet the quality standards of all ML use cases. Specific business objectives, as well as the risks associated with a low-performance model, will drive quality requirements. Some projects achieve goal quality using multiple annotators. Others require complex analysis against truth data or scaling workflows, with verification by an expert on the subject.
There are two main authority sources that can be used to measure the quality of feedback and are used to rate staff: gold data and expert review.
- Gold data: Gold data or ground truth sets can be used as a qualification tool to test and score staff at the beginning of the process, as well as as a measure of output quality. When you use gold data to measure quality, you compare employee ratings with expert ratings for the same data set, and the difference between these two blind and independent responses can be used to make quantitative measurements such as accuracy, recall, accuracy, and F1 scores. .
- Expert Review: This method of quality assurance is based on a review conducted by a highly qualified employee, administrator, or client expert, sometimes all three. It can be used in conjunction with Gold Data QA. The expert evaluator reviews the response provided by qualified personnel and approves or makes corrections accordingly, giving a new correct answer. Initially, expert review can be performed for each case of labeled data, but over time, as staff quality improves, expert review may use random sampling for ongoing quality control.
Repetition of data success
When a team of computer-based approaches has successfully launched a high-quality training data channel, they can move production toward a ready-made model. Through ongoing support, optimization and quality control, an external partner can help you:
- Track speed: To scale effectively, it is advisable to measure the performance of the comments. How long does it take to progress in the process? Is the process faster?
- Tune in to staff training: As the project scales progress, labeling and quality requirements can evolve. This requires ongoing staff training and scoring.
- Train edge cases: Over time, training data should have more and more cases to make your model as accurate and robust as possible.
Without high-quality training data, even the best funded and most ambitious ML / AI projects cannot be successful. Computer vision teams need reliable partners and platforms to provide the quality of data they need and to promote life-changing ML / AI models in the world.
Alegion is a proven partner in building a training data pipeline that will nurture your model throughout your life cycle. Contact Alegion solutions@alegion.com.
This content was created by Alegion. It was not written in the editorial board of the MIT Technology Review.
[ad_2]
Source link
