Paul Christiano

Capabilities and risks from frontier AI

On Wednesday, the U.K. government released a report called “Capabilities and risks from frontier AI,” in which it explains frontier AI as “highly capable general-purpose AI models that can perform a wide variety of tasks and match or exceed the capabilities present in today’s most advanced models.”

that the criteria apply,

that maintains horizontal exemption conditions and largely overlooks the negative legal opinion.

harshly criticised by the European Parliament’s legal office

under a pre-set list of critical use cases were deemed automatically high-risk

I usually always thought of hackers

Inclusive Framework on BEPS

The GloBE rules provide for a co-ordinated system of taxation intended to ensure large MNE groups pay this minimum level of tax on income arising in each of the jurisdictions in which they operate. The rules create a “top-up tax” to be applied on profits in any jurisdiction whenever the effective tax rate, determined on a jurisdictional basis, is below the minimum 15% rate.

Can I capture my vision in a page? A paragraph? A word?

The European Cyber Resilience Act (CRA)

We emphasize that EU policymakers should consider strengthening deployment requirements for entities that bring foundation models to market to ensure there is sufficient accountability across the digital supply chain.

Open releases generally achieve strong scores on resource disclosure requirements (both data and compute), with EleutherAI receiving 19/20 for these categories.

the European Parliament adopted a draft of the Act by a vote of 499 in favor, 28 against, and 93 abstentions.

We are at the same juncture now with AI. Once again, a handful of competitive but ideologically aligned leaders are telling us that large-scale, general-purpose AI implementations are the only way forward. In doing so, they disregard the dangerous level of complexity and the undue level of control and financial return to be granted to them.

What is happening is not just a battle for market control. A small number of tech titans are busy designing our collective future, presenting their societal vision, and specific beliefs about our humanity, as theonly possible path. Hiding behind an illusion of natural market forces, they are harnessing their wealth and influence to shape not just productization and implementation of AI technology, but also the research.

sraeli forces were still bombing Gaza and fighting with Hamas gunmen in parts of southern Israel in the early hours of Sunday and a spokesman for the military said the situation in the country was not totally under control.

Large-scale compute is also environmentally unsustainable: chips are highly toxic to produce17 and require an enormous amount of energy to manufacture:18 for example, TSMC on its own accounts for 4.8 percent of Taiwan’s national energy consumption, more than the entire capital city of Taipei.19 Running data centers is likewise environmentally very costly: estimates equate every prompt run on ChatGPT to the equivalent of pouring out an entire bottle of water.20

access to compute—along with data and skilled labor—is a key component

Artificial intelligence (AI) is in the news every day and corporate strategies are evolving to adapt our businesses to AI use.

2 Senators Propose Bipartisan Framework for A.I. Laws

Inside effective altruism, where the far future counts a lot more than the present

The Reluctant Prophet of Effective Altruism

ChatGPT-maker OpenAI and The Associated Press said Thursday that they’ve made a deal for the artificial intelligence company to license AP’s archive of news stories.

“It’s like asking, ‘should the newsroom use the Internet?’ in the 1990s,” Tofel said. “The answer is yes, but not stupidly.”

Restore the Index by Importing the Mapping

Korisnicima treba

New Delhi’s decision reflected its “growing concern at the interference of Canadian diplomats in our internal matters and their involvement in anti-India activities”, the ministry said in a statement on Tuesday.

Timeline of Systematic Data and the Development of Computable Knowledge

I think there is fatigue. If you ask the average New Yorker what the SDGs are, I’m not sure they’re going to be able to respond.

the hardening divides between the West vs. the global South, with the two camps mainly feuding over the reform of such financial institutions as the World Bank and the International Monetary Fund.

Only 15 percent of all 17 goals have been met, and 48 percent are off track and have either stagnated or regressed.

he 80th Session of the United Nations General Assembly, a High-Level Event on Science, Technology and Innovation for Development

the role of multi-stakeholder partnerships to foster strategic long-term investment in supporting the development of science, technology and innovation in developing countries,

including the Global Digital Compact, aligned with the Sustainable Development Goals, should be considered, which should offer preferential access for developing countries to relevant advanced technologies

triangular cooperation projects

inequalities in data generation

We acknowledge that all technological barriers, inter alia, as reported by the IPCC, limit adaptation to climate change and the implementation of the National Determined Contributions (NDCs) of developing countries

We note the central role of Governments, with the active contribution from stakeholders from the private sector, civil society, academia and research institutions, in creating and supporting an enabling environment at all levels, including enabling regulatory and governance frameworks

the expansion of open-science models

the knowledge produced by research and innovation activities can have in designing better public policies

the Tunis Agenda and the Geneva Declaration of Principles and plan of action shall lay down the guiding principles for digital cooperation.

to ensure that the World Summit on the Information Society (WSIS+20) General Review process, the Global Digital Compact and the Summit of the Future contribute to, inter alia, the achievement of sustainable development and closing the digital divide between developed and developing countries.

close alignment between the World Summit on the Information Society process and the 2030 Agenda for Sustainable Development,

the full implementation of the 2030 Agenda and the Addis Ababa Action Agenda

has the potential to resolve and minimize trade-offs among the Goals and targets,

an open, fair, inclusive and non-discriminatory environment for scientific and technological development.

stakeholders

The bot is wonderful, he said, letting him speed through $10 tasks in a matter of minutes. When we spoke, he was having it rate another chatbot’s responses according to seven different criteria, one AI training the other.

here were listings for AI trainers with expertise in health coaching, human resources, finance, economics, data science, programming, computer science, chemistry, biology, accounting, taxes, nutrition, physics, travel, K-12 education, sports journalism, and self-help. You can make $45 an hour teaching robots law or make $25 an hour teaching them poetry.

One way the AI industry differs from manufacturers of phones and cars is in its fluidity. The work is constantly changing, constantly getting automated away and replaced with new needs for new types of data. It’s an assembly line but one that can be endlessly and instantly reconfigured, moving to wherever there is the right combination of skills, bandwidth, and wages.

This debate spilled into the open earlier this year, when Scale’s CEO, Wang, tweeted that he predicted AI labs will soon be spending as many billions of dollars on human data as they do on computing power; OpenAI’s CEO, Sam Altman, responded that data needs will decrease as AI improves.

Taskup.ai, DataAnnotation.tech, and Gethybrid.io all appear to be owned by the same company: Surge AI. Its CEO, Edwin Chen, would neither confirm nor deny the connection, but he was willing to talk about his company and how he sees annotation evolving.

Often their work involved training chatbots, though with higher-quality expectations and more specialized purposes than other sites they had worked for.

“If there was one thing I could change, I would just like to have more information about what happens on the other end,” he said. “We only know as much as we need to know to get work done, but if I could know more, then maybe I could get more established and perhaps pursue this as a career.”

One engineer told me about buying examples of Socratic dialogues for up to $300 a pop. Another told me about paying $15 for a “darkly funny limerick about a goldfish.”

But if you want to train a model to do legal research, you need someone with training in law, and this gets expensive.

to be looking at their accuracy, helpfulness, and harmlessness

The model is still a text-prediction machine mimicking patterns in human writing, but now its training corpus has been supplemented with bespoke examples, and the model has been weighted to favor them

Each time Anna prompts Sparrow, it delivers two responses and she picks the best one, thereby creating something called “human-feedback data.”

“I remember that someone posted that we will be remembered in the future,” he said. “And somebody else replied, ‘We are being treated worse than foot soldiers. We will be remembered nowhere in the future.’ I remember that very well. Nobody will recognize the work we did or the effort we put in.”

Training a large model requires an enormous amount of annotation followed by more iterative updates, and engineers want it all as fast as possible so they can hit their target launch date.

According to workers I spoke with and job listings, U.S.-based Remotasks annotators generally earn between $10 and $25 per hour, though some subject-matter experts can make more. By the beginning of this year, pay for the Kenyan annotators I spoke with had dropped to between $1 and $3 per hour.

nstruction writers must come up with rules that will get humans to categorize the world with perfect consistency. To do so, they often create categories no human would use.

When AI comes for your job, you may not lose it, but it might become more alien, more isolating, more tedious.

coherent processes broken into tasks and arrayed along assembly lines with some steps done by machines and some by humans but none resembling what came before.

“AI doesn’t replace work,” he said. “But it does change how work is organized.”

A recent Google Research paper gave an order-of-magnitude figure of “millions” with the potential to become “billions.”

Annotation is big business. Scale, founded in 2016 by then-19-year-old Alexandr Wang, was valued in 2021 at $7.3 billion, making him what Forbes called “the youngest self-made billionaire,” though the magazine noted in a recent profile that his stake has fallen on secondary markets since then.

Mechanical Turk and Clickworker

CloudFactory,

Human intelligence is the basis of artificial intelligence, and we need to be valuing these as real jobs in the AI economy that are going to be here for a while.”

The more AI systems are put out into the world to dispense legal advice and medical help, the more edge cases they will encounter and the more humans will be needed to sort them.

Machine-learning systems are what researchers call “brittle,” prone to fail when encountering something that isn’t well represented in their training data.

The resulting annotated dataset, called ImageNet, enabled breakthroughs in machine learning that revitalized the field and ushered in a decade of progress.

The anthropologist David Graeber defines “bullshit jobs” as employment without meaning or purpose, work that should be automated but for reasons of bureaucracy or status or inertia is not.

But behind even the most impressive AI system are people — huge numbers of people labeling data to train it and clarifying data when it gets confused.

Like most of the annotators I spoke with, Joe was unaware until I told him that Remotasks is the worker-facing subsidiary of a company called Scale AI, a multibillion-dollar Silicon Valley data vendor that counts OpenAI and the U.S. military among its customers

oe got a job as an annotator — the tedious work of processing the raw information used to train artificial intelligence.

LLMs are through and through based on language and patterns to be found through it.

the very success of LLMs in the commonsense arena strongly suggests that you don’t fundamentally need deep “structured logic” for that.

One of the surprises of LLMs is that they often seem, in effect, to use logic, even though there’s nothing in their setup that explicitly involves logic.

“symbolic discourse language”

particular formal system that described certain kinds of things

we weren’t trying to use just logic to represent the world, we were using the full power and richness of computation.

Lenat–Haase representation-languages

the problem of commonsense knowledge and commonsense reasoning.

heuristics: strategies for guessing how one might “jump ahead”

a very classic approach to formalizing the world

Encode knowledge about the world in the form of statements of logic.

was it all just an “engineering problem” that simply required pulling together a bigger and better “expert system”?

to use the framework of logic—in more or less the same form that Aristotle and Leibniz had it—to capture what happens in the world.

She rejects notions of progress, she is despairing of representative democracy, and she is not confident that freedom can be saved in the modern world.

Like HTML, PDF facilitates the user’s choice of device and operating system. Unlike HTML, PDF does not assume that remote servers or content are available.

From the vantage-point of 2023 we are positioned to recognize 1993 as a year of two key developments; the first specification of HTML, the language of the web, and the first specification of PDF, the language of documents.

the plugin “Permalink Manager Lite” to manage the URL of the pages. 

In April 2020, I ran a small SEO experiment with a blog post and transformed a long-form article into a topic cluster (also called content hub). 

Strengthen telecommunications and data transfers thanks to a new undersea cable connecting the region.

How soon could AI replace human workers?

a soul-crushing amount of change and uncertainty — is to methodically plan for the future.

how — and when — their workforce will need to change in order to leverage AI.

The software includes more than 500 functions — but the vast majority of people only use a few dozen, because they don’t fully understand how to match the enormous number of features Excel offers to their daily cognitive tasks.

Rather, they’ll need to learn how to leverage multimodal AI to do more, and better, work

Most workers won’t need to learn how to code, or how to write basic prompts, as we often hear at conferences.

so that both the human and the AI can accomplish more through collaboration than by working independently.

She might go back and forth a few times, using different data sources, until an optimal quote is received for both the insurance company and the customer.

near- and long-term scenarios for the myriad ways in which emerging tools will improve productivity and efficiency

That’s because AI systems aren’t static; they are improving incrementally over time.

aren’t planning for a future that includes an internal RHLF unit tasked with continuously monitoring, auditing, and tweaking AI systems and tools.

Essentially, AI systems need constant human feedback, or they run the risk of learning and remembering the wrong information.

By marketing their platforms to companies, they want to lock them (and their data) in.

Business data is invaluable because once a model has been trained, it can be costly and technically cumbersome to port those data over to another system.

AI is not a monolith, and we are just at the beginning of a very long trajectory.

it’s not good enough to actually use.

This happened again in 1987, when again, computer scientists and businesses made bold promises on a timeline for AI that was just never feasible.

AI cycles through phases that involve breakthroughs, surges of funding and fleeting moments of mainstream interest, followed by missed expectations and funding clawbacks.

using an iterative process to cultivate a ready workforce, and most importantly, creating evidence-backed future scenarios that challenge conventional thinking within the organization.

The workforce will need to evolve, and workers will have to learn new skills, iteratively and over a period of years

leaders are focused too narrowly on immediate gains, rather than how their value network will transform in the future

the output has to be proven trustworthy, integrated into existing workstreams, and managed for compliance, risk, and regulatory issues.

Exactly which jobs AI will eliminate, and when, is guesswork.

Within just a few years, powerful AI systems will perform cognitive work at the same level (or even above) their human workforce.

They all want to know how their companies can create more value using fewer human resources.

How soon could AI replace human workers?

To my mind, the true spiritual forefather of AI was Gottfried Wilhelm von Leibniz (1646 - 1716), with his ideas about a universal formal language that could encompass all of human knowledge. Another important figure in AI's pre-history is Ada Lovelace, who around 1850 imagined that Charles Babbage's Analytical Engine (which unfortunately was never actually built) could conceivably accomplish such tasks as playing chess and composing music.

the Code of Conduct for Information Integrity on Digital Platforms that is being developed will be important.

There is convergence around the potential for a GDC to promote digital trust and security and to address disinformation, hate speech and other harmful online content.

There is broad consensus that the Internet Governance Forum (IGF) plays – and should continue to play – a key role in promoting the global and interoperable nature and governance of the Internet. The important roles played by IGF, ITU, UNCTAD, UNDP, UNESCO, WSIS and other UN entities, structures, and forums have been emphasized and that a GDC should not duplicate existing forums and processes.

OpenAI models leaned left/libertarian, Google's BERT conservative, Meta's LLaMA right-authoritarian.

Until the Kenyan government suspended the process, thousands of Kenyan citizens lined up to have their iris scanned using the Worldcoin orb. The amount of Worldcoin being offered to each person was estimated to be about $49.

We recognized this problem back in 2021 and argued that benchmarks should be as dynamic as the models they evaluate. To make this possible, we introduced a platform called Dynabench for creating living, and continuously evolving benchmarks. As part of the release, we created a figure that showed how quickly AI benchmarks were “saturating”, i.e., that state of the art systems were starting to surpass human performance on a variety of tasks.

Specifically, Google hopes to unify and standardize the evaluation metrics for unlearning algorithms, as well as foster novel solutions to the problem.

to identify problematic datasets, exclude them and retrain the entire model from scratch

OpenAI, the creators of ChatGPT, have repeatedly come under fire regarding the data used to train their models. A number of generative AI art tools are also facing legal battles regarding their training data.

The world doesn’t have alignment. How can AI.

You don't just put this stuff in public when it's within striking distance of achieving ASI. That's just insanely stupid. Don't compare it to ANY previous technology. We want to limit to as little rusk as possible.

On the surface it might seem like a bad thing to distribute dangerous technology, but the alternative is the lack of balance.

Even the transformer architecture was invented and released by Google researchers,

Everything they touched turned into a monopoly.

Are there any resources for educators to learn more about AI?

their ability to interact with AI

support individual growth.

how their skills in asking questions, analyzing responses, and integrating information have developed.

students can reflect on their progress over time.

to review each other's work, fostering a collaborative environment.

to observe critical thinking and problem-solving approaches.

Even if these tools could accurately identify AI-generated content (which they cannot yet), students can make small edits to evade detection.

To elaborate on our research into the shortcomings of detectors, one of our key findings was that these tools sometimes suggest that human-written content was generated by AI.

The goal is to help them “understand the importance of constantly working on their original critical thinking, problem solving and creativity skills.”

ecommends teachers use ChatGPT as an assistant in crafting quizzes, exams and lesson plans for classes.

She says exploring information in a conversational setting helps students understand their material with added nuance and new perspective.

But we don’t know how fast it’s moving. We don’t know why it’s working when it’s working.

the story here really is about the unknowns.

We can’t look at how a person thinks and explain their reasoning by looking at the firings of the neurons.

Trying to build systems where by design, every piece of the system means something that we can understand.

Interpretability is this goal of being able to look inside our systems and say pretty clearly with pretty high confidence what they’re doing, why they’re doing it.

The paper describing GPT-4 talks about how when they first trained it, it could do a decent job of walking a layperson through building a biological weapons lab.

they’ve just got to wait and see.

hey’re basically investing in a mystery box?

But that doesn’t mean we understand anything about the biology of that tree. We just kind of started the process, let it go, and try to nudge it around a little bit at the end.

There’s no explanation in terms of things like checkers moves or strategy or what we think the other player is going to do

steering these things almost completely through trial and error.

by just putting these systems out in the world and seeing what they do.

we don’t know how to steer these things or control them in any reliable way

We just don’t understand what’s going on here. We built it, we trained it, but we don’t know what it’s doing.