Logistics and Agenda

Homework 3 due Monday Sept 21 at 11:59pm

Agenda

• This week:  Data reduction & fitting, AI tools
• Next week:  Root-finding, numerical integration

Today

• AI tools demo
• AI overview and best practices
• Practical applications to fitting

Homework 3:

You may use AI freely to write code, debug, and test.

• We expect you to understand what the code is doing.

• You are responsible for the code you submit.

You may not use AI to write the report.

As with Homework 2:

• You may run unit tests in Gradescope to confirm correct execution

• You may resubmit as many times as you like until the deadline

Tech Demo

A quick tour of common AI-assisted workflows:

• Inline queries and editing
• Code explanation and summary
• Code critique
• Implementation
• Debugging
• Unit tests

Why use AI tools?

AI tools can reduce the time spent on routine implementation and help us focus on higher-level scientific questions.

• They can accelerate coding, debugging, documentation, and testing.
• They can help us explore unfamiliar code and libraries.
• They do not remove the need to understand the underlying physics, algorithms, or assumptions.

Goal:  Use AI to augment our computational workflow while retaining responsibility for the scientific result.

Levels of abstraction

Computing has repeatedly moved toward higher-level interfaces:

• Natural-language interfaces
• High-level languages such as Python
• Compiled languages such as C++
• Human-readable assembly
• Machine instructions
• Transistor logic

Source: generated with ChatGPT

Each layer hides some implementation details while making more complex tasks easier to express.

AI changes the workflow

Historically, a large fraction of scientific programming involved implementation details:

• Finding the right library or function (or writing it from scratch!)
• Converting data into the expected format
• Writing boilerplate code
• Diagnosing syntax and API errors

AI tools can help with much of this work.

This shifts some of our effort from writing code toward directing, reading, critiquing, and testing code.

You are still responsible for the result

AI-generated code should be treated like code written by someone else:

• Read it before using it.
• Understand what it is doing.
• Test it on cases where you know the answer.
• Check physical assumptions, units, limits, and conventions.
• Verify that the numerical result answers the scientific question you intended to ask.

A program can run successfully and still implement the wrong model.

Fundamental concepts — LLMs and inference

At a high level, a large language model (LLM) generates text by predicting a sequence of tokens from the information available in its context.

• Training: model parameters are adjusted using large datasets and substantial computation.

Source: generated with ChatGPT

• Inference: a trained model uses fixed parameters to generate a response.

Source: generated with ChatGPT

Model outputs can vary between runs. Different models have different strengths and limitations.

Fundamental concepts — context

The model does not automatically know everything that you know.

Context is the information available to the model for the current task.

This may include:

• Your prompt
• Earlier conversation
• Code or files you provide
• Tool outputs
• Documentation or other retrieved information

Good context is relevant context, not necessarily the maximum amount of context.

Fundamental concepts — limitations and tools

Important terms:

• Context window: the amount of information the model can consider at once.
• Attention decay / context rot: gradual degradation in performance as the context exceeds the context window.
• Rate limits: restrictions on how often or how much you can use a model.
• Hallucination: a plausible-sounding but incorrect or unsupported response.
• Tools: capabilities that let the model interact with files, a terminal, documentation, or other resources.
• Agent: an LLM that can use tools repeatedly as part of an iterative multi-step workflow.

Tool access increases capability, but also increases the importance of reviewing what the model is about to do.

Common use cases

Give precise instructions

Good prompts clearly specify the task, relevant context, and constraints.

For example:

• State what the code is supposed to do.
• Identify the files or functions that matter.
• Specify what should not be changed.
• Ask the model to use the relevant documentation or API.
• Define how you will decide whether the result is correct.

Precise instructions make the model easier to evaluate and reduce unnecessary changes.

Separate planning from implementation

For nontrivial tasks, work in stages:

explain → suggest → plan → implement → validate

This makes it easier to catch a bad assumption before it becomes a large code change.

For scientific computing, validation should include both:

• Software verification: did we implement the algorithm correctly?
• Scientific validation: is this the correct model for the physical problem?

Deterministic code and non-deterministic AI

Use each tool for what it does well.

• LLMs: generation, explanation, planning, and proposing alternatives
• Code: reproducible calculations, automated tests, and quantitative validation

AI can make it easier to write tests, but you must still decide what constitutes convincing evidence that the calculation is correct.

AI Safety

These tools are very powerful, but they come with serious risks.

With great power comes great responsibility

Stan Lee (1962)

• Assume that everything you type or upload to an external AI service will be known to the world
  • Except models that are locally run (on your computer or on W&M HPC)
  • Some providers ‘promise’ that data will not be used (but how can we check it?)
• Agents: if you run an agent (permit local code execution) from an AI service, all bets are off
  • You are trusting an external company to do no evil
  • There is no guarantee that safety features are bug free
  • It is best to run agents in containers or virtual machines where they are in lock down and cannot access your files outside of a closed directory
• Unfortunately, the safer it is the less convenient it is to work with

Access and navigation

Note: You must be on campus or connected through the VPN.

Navigation

• Chat window
• Chat organization
• Settings
• Modification history and diff view

Context and commands

Use context controls to tell the model what information matters for the current task.

• #: select or reference relevant context
• /: invoke commands or actions
• @: reference available resources or tools, where supported

The exact behavior depends on the interface, so pay attention to what information is actually being shared with the model.

Interaction modes

Different modes provide different levels of autonomy.

• Ask
  • Read-only interaction
  • Useful for explanation, summary, and critique
• Plan
  • Proposes a sequence of changes without implementing them
• Agent
  • Can carry out multi-step tasks using available tools

Use the least-permissive mode that is sufficient for the task.

Tool permissions

Before approving a tool action, understand what the model is asking to do.

Examples:

• Inline editing
  • Review the diff before accepting changes.
• Running code
  • Check the command before allowing terminal access.
• Modifying files
  • Confirm which files will change.
• Writing tests
  • Check that the tests actually probe the intended behavior.

Stop the agent if its actions no longer match your intent.

Fitting demo workflow

Apply the workflow to the fitting example from the previous lecture:

1. Ask for a high-level explanation of the fitting code.
2. Ask for a critique of its assumptions and failure modes.
3. Ask for a plan to improve the code.
4. Review the proposed plan.
5. Implement one change at a time.
6. Run the code and tests.
7. Validate the fitted result against known limits or synthetic data.

Rules of thumb

• Understand requests before approving tool use.
• The model only sees the context you provide.
• Good context is relevant, specific, and manageable.
• Give precise instructions, including constraints and non-goals.
• Separate planning from implementation for larger tasks.
• Review diffs and terminal commands before approving them.
• Use deterministic code and tests to check AI-generated work.
• Working code is not necessarily scientifically valid.

Bottom line:  AI can accelerate scientific computing, but verification and scientific judgment remain your responsibility.