Browsed by
Tag: Deep learning

State of AI 2020 – Article by Pavel Ilin

State of AI 2020 – Article by Pavel Ilin

logo_bg

Pavel Ilin


This summary is prepared based on the State of AI Report 2020, which was crafted by Nathan Benaich and Ian Hogarth.

The AI industry is very diverse in its application, and it’s going through a transformation from the magical-wand stage to the plateau of adequate development. Let’s take a look at what is happening in the AI industry.

Research

We haven’t come up with new super-smart algorithms. Progress in model performance keeps being driven by big computational budgets and huge data sets. Training of the GPT-3 language model, with its 175 billion parameters, cost approximately $10 million. At the same time larger models require less data to achieve the same level of performance. With a deep-learning approach we are getting close to the point when the cost of training will grow outrageous with incrementally smaller improvements of the model.

An important fact is that the code base of most artificial intelligence systems remains closed. Only 15% of papers publish their code. This raises a lot of concerns about reproducibility and AI safety. AI explainability remains a critical issue for AI safety research; there are promising avenues of exploration such as Asymmetric Shapley Values, but so far it’s unknown how AI systems make decisions. 

Natural language processing (NLP) models successfully simulate common scenes and linguistics, but they fail dramatically with understanding problems and context and forming knowledge. 

Talent

Talented people with skills in math and computer science are the drivers of the progress in the AI field. More and more US professors are being recruited by tech companies. This affects the quality of education that US universities can provide. We already can see a decline in the level of entrepreneurship among recent graduates. At the same time Universities are creating AI-related degree programs.

The US keeps its position as the main attractor of talented individuals. For example China contributes to the talent pool of AI developers, but after publication of their first results, talented people are most likely to move to the US. 90% of international PhD graduates stay and work in US universities and corporations. Demand for AI talent remains much higher than supply, even despite COVID-19’s impact on market growth.  

Industry

AI keeps progressing not only on a theoretical and research level. Many real world applications are already in use, and they are affecting the industries in various ways.

New drugs are being designed by AI, and they are already in clinical trials. For example AI-designed drugs for OCD treatment are out for testing in Japan. AI drug-discovery startups keep raising funds. Also big pharma is teaming up with startups around preserving privacy during drug discovery. For example OpenMined uses federated learning to preserve privacy with medical data. Viz.ai presented the first product which was approved by the Centers for Medicare and Medicaid Services in the US. Their product analyzes tomography scans and alerts specialists who can treat patients before they receive damage that leads to the long-term disability. 

Progress in self-driving cars stays limited. Only 3 companies in California have permission to conduct testing of self-driving cars without a safety driver. Self-driving mileage remains microscopic compared to human drivers (2,874,950 miles for self-driving cars versus 390,313,739,000 miles for humans). The research and development process for self-driving cars remains very expensive. The major companies in this field raised around $7 billion since July 2019. Tesla chose to approach gradually adding self-driving features to its cars, but human drivers still remain in the loop. Recent approaches such as supervised learning do not perform well enough. To make dramatic breakthroughs, new approaches are required.

Computer vision unlocks faster accident and disaster recovery intervention. It also reduces the amount of human hours spent using a microscope, which could lead to acceleration of development processes and reduction of product costs.

AI drives sales and at the same time reduces costs in supply chains and manufacturing. Robotic process automation and computer vision are the most commonly deployed techniques in the enterprise. Speech, natural language generation, and physical robots are the least common. Recently IBM partnered with health insurance company Humana. IBM implemented natural language understanding (NLU) software which is already live and handles calls. It not only redirects calls to the different queues; it’s able to answer basic questions, such as “How much will the copay be to visit a specific specialist?” without human intervention.

Modern AI, in order to perform well, requires a lot of computing resources. Specialized AI hardware keeps progressing, and companies are now presenting second generations of their products. Graphcore M2000 offers faster training time to drop the cost of state-of-the-art models. Google’s new TPU v4 delivers up to a 3.7x training speedup over their TPU v3. NVIDIA will not rest either; it has achieved up to 2.5x training speedups with the new A100 GPU vs V100. Increasing interest towards machine learning devOps is a signal that the industry shifting its focus from how to build models to how to run them.

Despite the COVID-19 pandemic, investments keep coming into the industry. Private funding rounds of greater than $15 million for the AI-first companies remain strong.

Politics

Usage of AI for facial recognition tasks is extremely common around the world. Around half of the world allows facial recognition. This has become a recognizable political and ethical problem, especially when use of this technology leads to the wrong arrests. There were two highly publicized cases of wrong arrest in the US (which is probably just a tip of the iceberg). In May 2019, Detroit police arrested Michael Oliver who was wrongly accused of a felony for supposedly reaching into a teacher’s vehicle, grabbing a cellphone and throwing it, cracking the screen, and breaking the case. In January 2020, Detroit police arrested Robert Williams as a shoplifter who allegedly stole five watches from Midtown’s trendy Shinola store in October 2018. In both cases charges were dismissed but harm was done. 

Industry took a more thoughtful approach as a reaction to the AI mistakes. Microsoft deleted its database of 10 million faces, Amazon announced a one-year pause on letting the police use its facial recognition tool Rekognition. IBM announced it would sunset its general purpose facial recognition products. Washington State in the US introduced requirements to acquire warrants to run facial recognition scans. The ImageNet, a popular image database, is making an effort toward reduction of the biases in its image collections.

As Deep Fake technology produces more and more realistic media, it becomes illegal to use in certain states in the US. California passed a law, AB 730, aimed at deep fakes, which criminalizes distributing audio or video that gives a false, damaging impression of a politician’s words or action. Many other US state bills have been passed, addressing different risks. For example Virginia law amends current criminal law on revenge porn to include computer-generated pornography.

The US government keeps pursuing implementation of the military AI systems. DARPA organised a virtual dogfighting tournament where various AI systems would compete with each other and a human fighter pilot from the US military.

AI nationalism is on the rise. Countries tend to pursue protectionist policies to scrutinize acquisitions of AI companies by the players from other countries.

Every year AI plays a more and more noticeable part in our lives. It becomes cheaper, and you learn how to do new things. But we have to remember that at the moment AI is still a tool. And there are some philosophical and methodological difficulties which we have to overcome before it will be possible to deliberate about the potential sentience of the AI. It’s very important for the policy makers to make informed decisions based on how technology actually works and not on magical understanding formed based on popular sci-fi.

Pavel Ilin is Secretary of the United States Transhumanist Party. 

 

Scientists Identify Genes Implicated in the High Regenerative Capacity of Embryos and ESCs – Press Release by Biogerontology Research Foundation

Scientists Identify Genes Implicated in the High Regenerative Capacity of Embryos and ESCs – Press Release by Biogerontology Research Foundation

Biogerontology Research Foundation


CREDIT: AGEX THERAPEUTICS, INSILICO MEDICINE & THE BIOGERONTOLOGY RESEARCH FOUNDATION

Below is a press release by Biogerontology Research Foundation on the regenerative capacity of embryos and embryonic stem cells. This press release was originally published here.

~ Kenneth Alum, Director of  Publication, U.S. Transhumanist Party, January 18, 2018

 

Friday, January 12, 2018, London, UK: Researchers at Insilico MedicineAgeX Therapeutics and the Biogerontology Research Foundation have published a landmark study titled “Use of deep neural network ensembles to identify embryonic-fetal transition markers: repression of COX7A1 in embryonic and cancer cells” in the journal Oncotarget.

In the study, researchers used deep-learning techniques to analyze gene expression data in embryonic stem cell (ESC) lines at varying stages of development in order to characterize the gene expression profile of cells right at the boundary of the embryonic-fetal transition, when embryos become fetuses and experience a remarkable reduction in their regenerative capacity. In essence, the study’s objective was to hone in on those genes responsible for the remarkable regenerative capacities of embryos and ESCs.

“This is another important step in the progress of Insilico Medicine and indicates that its suite of products is developing rapidly, with significant commercial revenues not far off,” said Jim Mellon, Trustee of the Biogerontology Research Foundation, Chairman of Juvenescence Limited and a key partner of Insilico Medicine.

Mimicking the gene expression profile of cells prior to the embryonic fetal transition in adult tissues and organs is the concept underlying one of the central and most ambitious therapeutic modalities being pursued by AgeX Therapeutics, namely induced Tissue Regeneration (iTR). Therapeutic elaboration of the insights derived from this study could pave the way for in-situ tissue regeneration, and its application to ageing and age-related disease.

“induced Tissue Regeneration (iTR) is one of the most promising therapeutic modalities for enabling in-situ tissue regeneration proposed to date, and one that is likely to bring substantial healthspan-extending effects if implemented. This landmark study paves the way toward that bright future. Interestingly, in its identification of COX7A1 as one of the genes implicated in the remarkable regenerative potential of embryos and ESCs, the study also extends the purview of these findings to novel potential cancer therapies as well,” said Franco Cortese, Deputy Director of the Biogerontology Research Foundation.

The authors also developed effective methods of deriving biologically-relevant information from these profiles, identifying the most interesting genes characterizing the regenerative capacity of ESCs, and performed additional experimental validation to support the findings of the study’s deep learning analysis. Interestingly, one of the genes implicated in the embryonic-fetal transition that the study identified, COX7A1, is dysregulated in a diverse array of cancer types, including breast, lung, kidney, bone and muscle. As such, the results of this study could be used create novel cancer therapies as well.

“AI is quickly becoming the main driver of progress in so many fields of science, technology and human endeavor that it is easy for one to lose count. From healthcare to finance to governance, AI is galvanizing rapid paradigm shifts all around us. Insilico Medicine is rapidly establishing themselves as the leader of AI for longevity, and the combination of their deep-learning expertise with the assets for expert experimental validation and interpretation possessed by AgeX Therapeutics is a partnership that has yielded significant synergistic results in using AI to yield novel insights into the biology of aging and charting the path toward next generation healthspan-extending therapies” said Dmitry Kaminskiy, Managing Trustee of the Biogerontology Research Foundation.

###

Paper Reference: West M, Labat I, Sternberg H, Larocca D, Nasonkin I, Chapman K, Singh R, Makarev E, Aliper A, Kazennov A, Alekseenko A, Shuvalov N, Cheskidova E, Alekseev A, Artemov A, Putin E, Mamoshina P, Pryanichnikov P, Larocca J, Copeland K, Izumchenko E, Korzinkin M and Zhavoronkov A. Use of deep neural network ensembles to identify embryonic-fetal transition markers: repression of COX7A1 in embryonic and cancer cells, Oncotarget. 2017; in press, https://doi.org/10.18632/oncotarget.23748

About the Biogerontology Research Foundation:

The Biogerontology Research Foundation is a UK non-profit research foundation and public policy center seeking to fill a gap within the research community, whereby the current scientific understanding of the ageing process is not yet being sufficiently exploited to produce effective medical interventions. The BGRF funds and conducts research which, building on the body of knowledge about how ageing happens, aims to develop biotechnological interventions to remediate the molecular and cellular deficits which accumulate with age and which underlie the ill-health of old age. Addressing ageing damage at this most fundamental level will provide an important opportunity to produce the effective, lasting treatments for the diseases and disabilities of ageing, required to improve quality of life in the elderly. The BGRF seeks to use the entire scope of modern biotechnology to attack the changes that take place in the course of ageing, and to address not just the symptoms of age-related diseases but also the mechanisms of those diseases.

About Insilico Medicine, Inc.:

Insilico Medicine, Inc. is a bioinformatics company located at the Emerging Technology Centers in Baltimore with R&D resources in 6 countries. The company is widely recognized by the industry for applying next-generation artificial intelligence technology to drug discovery and aging research. For its pioneering work in the applications of Generative Adversarial Networks (GANs) and Reinforcement Learning (RL) and collaborations with the pharmaceutical companies, it was selected as one of the Top 100 AI companies 2018 by CB Insights and Top 5 AI companies for social impact 2017 by NVIDIA. The company pursues internal drug discovery programs in cancer, dermatological, metabolic and CNS diseases, sarcopenia, fibrosis and senescence. Company website: http://www.insilico.com

About AgeX Therapeutics:

AgeX Therapeutics, Inc., a subsidiary of BioTime, Inc. (NYSE American: BTX), is a biotechnology company applying technology relating to cellular immortality and regenerative biology to aging and age-related diseases. The company has three initial areas of product development: pluripotent stem-cell-derived brown adipocytes (AGEX-BAT1); vascular progenitors (AGEX-VASC1); and induced Tissue Regeneration (iTR). Initial planned indications for these products are Type 2 diabetes, cardiac ischemia, and tissue regeneration respectively. For more information, please visit http://www.agexinc.com or connect with the company on Twitter or Facebook.