The White House / Executive Order / Hyperscale AI 2023 Nov Week 1 Datumo Newsletter |
|
|
📰 U.S. Move Towards 'Hyperscale AI' Certification |
|
|
U.S. President Joe Biden signing the executive order
This Monday(Oct 30), President Joe Biden of the United States signed an executive order aimed at preventing the risks associated with AI technology.
The executive order, comprising 13 sections, encompasses various fields including employment, diplomacy, healthcare, judiciary, and administration. Despite its previously lenient stance on AI development and utilization compared to Europe, the U.S. is now being viewed as taking a lead in setting global standards.
An executive order holds a similar effect to legislation during a presidential term, and federal agencies can establish laws and regulations based on it. We explored the risks brought by AI and the trends in regulation in our newsletter on the third week of September (Big Tech Leaders Gather for 'AI Insight Forum').
This executive order is a result of the AI hearings held in the U.S. Congress forums, with a notable point being the government’s management and supervision over large-scale language model training and possession.
|
|
|
Sec 4. Ensuring the Safety and Security of AI Technology. |
|
|
Section 4.2 Ensuring Safe and Reliable AI.
Now, companies that develop or plan to develop AI model smart enough to significantly impact society must submit relevant information, reports, or records.
The criterion is the computing power utilized in model training. If the amount of computation required for model training exceeds 1e26 (10 to the power of 26) FLOPs (floating-point operations), it's subject to submission. Typically, the number of FLOPs increases in proportion to the number of parameters and the size of the training data.
So far, there has been no publicly disclosed model with training FLOP counts exceeding 1e26. The training FLOP count for Google’s ‘PaLM’, which trained 540 billion parameters with 780 billion tokens, is 2.56x1e24, and according to the data from Our World In Data (OWID), the training FLOP count for GPT-4 is 2.1x1e25. We do not yet know what kind of model surpasses this training criterion, which is about 5 times higher, nor do we know what level of performance it will show.
|
|
|
However, the figure 1e26 comes from a research paper titled "Frontier AI regulation: Managing emerging risks to public safety" published in July by OpenAI, along with researchers from Google DeepMind and the Centre for the Governance of AI.
In the paper, the researchers mention 1e26 as a benchmark for 'large models' that have the potential to cause substantial harm. It's a notable point to see how closely big tech companies and the White House are engaged in ongoing discussions regarding AI risk management.
"Perhaps more than any model that has been trained to date. Estimates suggest that 1E26 floating point operations (FLOP) would meet this criterion." Frontier AI Regulation: Managing Emerging Risks to Public Safety. Markus Anderljung et al.
|
|
|
'Example Frontier AI Lifecycle' Regulatory framework proposed in the July research.
Source: Frontier AI regulation: Managing emerging risks to public safety |
|
|
GPT-3.5-turbo is 20B parameter |
|
|
|
The Microsoft Research group, while authoring the paper "CodeFusion: A Pre-trained Diffusion Model for Code Generation," disclosed that the parameter size of GPT-3.5-turbo (which serves as the foundation for ChatGPT) is 20B (refer to the image above). The base model of GPT-3.5 has 175B parameters. Users(reddit) are astounded by OpenAI's technological prowess.
|
|
|
The Data for Smarter AI
Datumo is an all-in-one data platform for your smarter AI.
|
|
|
Datumo Inc.
📋 contact@datumo.com
|
|
|
|
|
