When Accuracy Isn't Enough

I completed two assessments this week and began to work on my digital portfolio.

The first was my Industry Trends and Future Outlook analysis of the machine learning industry. The statistic that really caught my eye was from the McKinsey survey. 80% of workers reported that AI has increased their productivity, but only 37% of companies reported any impact on their earnings, which remained unchanged from last year. Companies are investing more and more in AI, and many of them are failing to demonstrate the return on investment. Writing this changed my plans for what to study. I began ISM with the intent of learning how models are constructed. Now I'm more interested in why a model that works in testing doesn't work when it's deployed.

The annotated bibliography further drove me in that direction. My area of interest was ethics and deployment risks in the field of AI. The Gender Shades paper was the one that resonated the most. One commercial product was incorrect 0.8% of the time on lighter-skinned men and 34.7% of the time on darker-skinned women. I'm a vision coder for my robotics team, and I've never analyzed my test results by anything. If the overall accuracy passed, I went on to the next one. After reading this, I'm going to break down my test sets by lighting and distance so that a failure like that can't hide inside one average.

I began my digital portfolio as well. I've created the Home page and the About Me page so far. I still need to get the Résumé, About ISM, Mentor Bio, Research, Blog and Projects pages finished. My goal for next week is to have all eight tabs functioning so that each tab goes to its own page, though some may be relatively empty for now. When the Research page is up, I will include links to both assessments from this week.