Class goals and structure

Primary purpose

• learn to specify a problem
• break it up into algorithmic pieces
• implement a program to execute these pieces
  • learn Python and generative AI tools
• check validity of the implementation

Early history of computing

• Computers used to be humans
• Computing aids - no programming possible
  • abacus
  • sliding ruler
  • pre-calculated tables of function (logarithm, trigonometry …)
  • mechanical calculators
• Modern computers appear at 1946
  • ENIAC (Electronic Numerical Integrator And Computer)

Two women operating ENIC. Source Wikipedia.

• weight: 30 tons
• cost: $500,000 ($6,000,000 adjusted)
• power consumption: 150 kW

ENIAC vs modern PC

What can be done in 1 second?

Common features of modern computer

• Central Processing Unit (CPU)
• memory
  • holds data and executable code
• data input and output
• same hardware can do different calculation sequences
• usually use binary system
• programmable for any general task

Speed measured in FLOPS (the number of floating point operations per second) which usually proportional to the clock frequency.

Different computer architectures (AMD, Mac, Intel, ARM …) have different proportionality coefficient.

My 2 GHz AMD Phenon PC (circa 2011) can do about 50 MegaFLOPS

Energy consumption

Brain

• Idle: 100W
• Heavy use: 100W

Brain. Source Wikipedia

Desktop or Notebook

• Idle: < 100W
• Heavy use: 200W - 1000W

AI capable computer

• Idle: < 100W
• Heavy use: 1kW - 10kW

Data center

• Idle: NA
• Heavy use: town level

Amazon data center. Source Wikipedia

Note: Data centers annual energy use is $\approx 4.4$% of U.S. annual electricity consumption according to congress.gov (2023 report)

Computers …

Computers are incredibly fast, accurate, and stupid. Humans beings are incredibly slow, inaccurate, and brilliant. Together they are powerful beyond imagination.

Leo Cherne (1969)

Computer is not a substitute for a brain. Source AI.

Programming languages implementations

Python as a language of choice

Python was conceived by Guido van Rossum with the first release in 1991.

Python: where to get

• free for all
• available for pretty much any OS
• visit https://www.python.org/
• go to ‘Download’ and download
• make sure that download size is about 30 MB

Please, do it before this Friday class, also do not forget to bring your notebooks/laptops with you for Friday classes.

Binary representation - why PHYS 256

Modern general purpose computers use binary representation

• bit is a smallest unit of information
• bit value is either 0 or 1

Bit is too small so we use byte

• byte = 8 bits stitched together
• byte can represent $256 = 2^8$ different values
• the major (the left most) bit usually holds the sign ($s$) of the number
  • 0: means positive
  • 1: means negative
• example $11001010_2$ = $(-1)^1 \times (0 \times 2^0 +1 \times 2^1 + 0 \times 2^2 + 1 \times 2^3 + 0 \times 2^4 +0 \times 2^5 +1 \times 2^6)$ = $-(2 + 8 + 64)$ = $-74$
  • note: ones’ complement another common way
• binary representation is always subject of interpretation

Binary representation (cont.)

Byte is clearly too small to be used for real life computation.

Note that in stock Python integers are unbound

• for example 10**500 produces number with 500 zeros
• 1 + 10**500 or 2 * 10 **500 works
• but 10*500 / 2 fails

but in practical (fast) math Python uses up to 8 bytes or 64 bits for number representation (this is CPU dependent)

• available range $-9,223,372,036,854,775,808 \cdots 0 \cdots 9,223,372,036,854,775,807$
  • i.e. bounded by $\approx 10^{19}$
• you can find this range by executing sys.maxsize
• notice that if you attempt to use large numbers they will be converted to float representation
  • calculation time will increase

Floating-point representation aka scientific notation

What to do if you need to store a float number?

For example $-123.765 \times 10^{12}$

• First convert it to scientific notation

  • $-1.23765 \times 10^{14}$

• truncate it to certain number of significant digits

  • let’s use 4 for example (actually 17 decimals for 64 bits float number)
  • $-1.237 \times 10^{14}$

• resulting number should have a form $(-1)^s \times c \times b^{q}$

  • where $s$ is a sign bit (1 in our case)
  • c is mantissa or coefficient or fraction (1.237)
  • b is the base (10)
  • q is the exponent (14)

Floating-point binary representation

Computers internally use binary base

• resulting number should have a form $(-1)^s \times c \times b^{q}$
• $b=2$
• 64 bits for full representation
  • 52+1 bits for mantissa or coefficient or fraction (about 17 decimal digits)
  • 11 bits for exponent ($\pm 308$)

IEEE 754 internal binary representation. Source Wikipedia

Overflow and underflow errors

• maximum $\pm 1.7976931348623157 \times 10^{308}$
  (provided by sys.float_info.max in Python)

  • overflow error
    • $(1.7976931348623157 \times 10^{308}) \times 10 = \texttt{inf}$

• smallest non zero $\pm 2.2250738585072014 \times 10^{-308}$ (provided by sys.float_info.min in Python)

  • it is also calculated as $2^{-1022}$
  • you can work with a smaller number via so called subnormal numbers $2^{-1074} \approx 5 \times 10^{-324}$
    • it might be slower
  • underflow problem
    • $(2^{-1074}) / 2$ produces $0$

Round off errors

• round off or truncation error
  • $1.79769313486231\mathbf{ 6} + 20 \to 21.79769313486231\mathbf{8}$
  • $1.79769313486231\mathbf{6} + 100 \to 101.79769313486231\mathbf{\_}$
  • $1.797693134862\mathbf{316} + 1000 \to 1001.7976931348624\mathbf{\_\_\_}$
• how to mitigate
  • try to use numbers of the similar magnitude
  • do not trust the least significant digits