--- author: - "by [Eugeniy E. Mikhailov](http://physics.wm.edu/~evmik/) and [Greg Bentsen](https://physics.wm.edu/~gbentsen/)" title: Computers and programming languages introduction --- ## Class goals and structure Primary purpose - learn to specify a problem - break it up into algorithmic pieces - implement a program to execute these pieces - learn Python and generative AI tools - check validity of the implementation # Computers introduction ## Early history of computing - Computers used to be humans - Computing aids - no programming possible - abacus - sliding ruler - pre-calculated tables of function (logarithm, trigonometry ...) - mechanical calculators - Modern computers appear at 1946 - ENIAC (Electronic Numerical Integrator And Computer) ::: columns ::: {.column width="45%"} ![Two women operating ENIC. Source Wikipedia.](./pics/Two_women_operating_ENIAC.png){width="80%"} ::: ::: {.column width="45%"} - weight: 30 tons - cost: \$500,000 (\$6,000,000 adjusted) - power consumption: 150 kW ::: ::: ## ENIAC vs modern PC What can be done in 1 second? ::: columns ::: {.column width="45%"} ENIAC - 5000 additions - 357 multiplications - 38 divisions ::: ::: {.column width="45%"} Intel i7 3000+ (5GHz) circa of 2025 - 2,600,000,000 additions - 1,200,000,000 multiplications - 193,000,000 divisions - 146,000,000 sin operations ::: ::: ## Common features of modern computer - Central Processing Unit (CPU) - memory - holds data and executable code - data input and output - same hardware can do different calculation sequences - usually use binary system - programmable for any general task Speed measured in FLOPS (the number of floating point operations per second) which usually proportional to the clock frequency. Different computer architectures (AMD, Mac, Intel, ARM ...) have different proportionality coefficient. My 2 GHz AMD Phenon PC (circa 2011) can do about 50 MegaFLOPS ## Energy consumption ::: columns ::: {.column width="23%"} Brain - Idle: 100W - Heavy use: 100W ![Brain. Source Wikipedia](./pics/human_brain_vector.png){width="80%"} ::: ::: {.column width="23%"} Desktop or Notebook - Idle: \< 100W - Heavy use: 200W - 1000W ::: ::: {.column width="23%"} AI capable computer - Idle: \< 100W - Heavy use: 1kW - 10kW ::: ::: {.column width="23%"} Data center - Idle: NA - Heavy use: town level ![Amazon data center. Source Wikipedia](./pics/Amazon_AWS_us-west-2_morrow_east_AZ_01.jpg){width="80%"} ::: ::: Note: Data centers annual energy use is $\approx 4.4$% of U.S. annual electricity consumption [according to congress.gov](https://www.congress.gov/crs-product/R48646) (2023 report) ## Computers ... > Computers are incredibly fast, accurate, and `\alert{stupid}`{=tex}. > Humans beings are incredibly slow, inaccurate, and brilliant. Together > they are powerful beyond imagination. > > Leo Cherne (1969) ![Computer is not a substitute for a brain. Source AI.](./pics/computer_is_not_brain.png){width="80%"} # Programming languages overview There are hundreds of programming languages. - Super low-level language - binary code - the only thing which computers understand - each instruction looks like a number - usually it is not human readable - low-level languages - assembler (human readable binary code translation) - Fortran, LISP, C, C++, Forth - higher-level languages - Tcl, Java, JavaScript, PHP, Perl, Python - super-high-level - AI or LLM assisted `\alert{Unfortunately none of them serve all needs.}`{=tex} # Programming languages benchmark ::: columns ::: {.column width="45%"} ![Execution time. Smaller is better](./pics/ray_benchmark_time.png){width="80%"} ::: ::: {.column width="45%"} ![Memory usage. Smaller is better](./pics/ray_benchmark_memory.png){width="80%"} ::: ::: ## Programming languages implementations ::: columns ::: {.column width="33%"} **Compiled** - generate computers binary code - it takes time - faster execution time - a bit harder to debug - if you find and fixed an error (bug) you need to recompile - Examples: Assembler, C, C++, Fortran ::: ::: {.column width="33%"} **just-in-time compilation** - middle ground - compile once to bytecode before or during execution - cross-platform - Examples: Java, Python ::: ::: {.column width="33%"} **Interpreted** - No compilation - interpretation to machine code per instruction - slow (since you have to interpret same instruction over and over) - cross-platform code - Examples: Python, Perl, JavaScript, Lua, Php, Tcl, Shells, Matlab ::: ::: ## Python as a language of choice ::: columns ::: {.column width="45%"} **Pros** - interpreted - easy to use and debug - quite fast if done right, since main functions are compiled - `\alert{large selection of scientific related functions}`{=tex} - built in graphics and plotting - Turing complete (you can do with it everything which computer is capable) - rudimentary symbolic calculations ::: ::: {.column width="45%"} **Cons** - interpreted - could be slow if programmed inefficiently - numerical calculations are afterthought ::: ::: Python was conceived by Guido van Rossum with the first release in 1991. ## Python: where to get - free for all - available for pretty much any OS - visit - go to 'Download' and download - make sure that download size is about 30 MB `\alert{Please, do it before this Friday class}`{=tex}, also do not forget to bring your notebooks/laptops with you for Friday classes. # Discretization - the main weakness of computers - coming from resource limitation For example: $1/6=0.1666666666666666\cdots$ But computer has limited amount of memory. Thus it cannot hold infinite amount of digits and has to truncate somewhere. Let's say it can hold only 4 significant digits. $$1/6=0.1667_c$$ This called `\alert{round off error}`{=tex} due to truncation/rounding. Then for computer $$1/6=1/5.999$$ or $$0.\mathbf{1667}123 = 0.\mathbf{1667}321 = 0.\mathbf{1667}222 = 0.\mathbf{1667}111$$ or even more interesting $$20 \times (1/6)-20/6=20 \times 0.1667 - 3.333= 3.334-3.333=10^{-3}$$ # Binary representation ## Binary representation - why PHYS 256 Modern general purpose computers use binary representation - bit is a smallest unit of information - bit value is either 0 or 1 Bit is too small so we use byte - byte = 8 bits stitched together - byte can represent $256 = 2^8$ different values - the major (the left most) bit usually holds the sign ($s$) of the number - 0: means positive - 1: means negative - example $11001010_2$ = $(-1)^1 \times (0 \times 2^0 +1 \times 2^1 + 0 \times 2^2 + 1 \times 2^3 + 0 \times 2^4 +0 \times 2^5 +1 \times 2^6)$ = $-(2 + 8 + 64)$ = $-74$ - note: [ones' complement](https://en.wikipedia.org/wiki/Ones'_complement) another common way - binary representation is always subject of interpretation ## Binary representation (cont.) Byte is clearly too small to be used for real life computation. Note that in stock Python integers are unbound - for example `10**500` produces number with 500 zeros - `1 + 10**500` or `2 * 10 **500` works - but `10*500 / 2` fails but in practical (fast) math Python uses up to 8 bytes or 64 bits for number representation (this is CPU dependent) - available range $-9,223,372,036,854,775,808 \cdots 0 \cdots 9,223,372,036,854,775,807$ - i.e. bounded by $\approx 10^{19}$ - you can find this range by executing `sys.maxsize` - notice that if you attempt to use large numbers they will be converted to float representation - calculation time will increase ## Floating-point representation aka scientific notation What to do if you need to store a float number? For example $-123.765 \times 10^{12}$ - First convert it to scientific notation - $-1.23765 \times 10^{14}$ - truncate it to certain number of significant digits - let's use 4 for example (actually 17 decimals for 64 bits float number) - $-1.237 \times 10^{14}$ - resulting number should have a form $(-1)^s \times c \times b^{q}$ - where $s$ is a sign bit (1 in our case) - c is mantissa or coefficient or fraction (1.237) - b is the base (10) - q is the exponent (14) ## Floating-point binary representation Computers internally use binary base - resulting number should have a form $(-1)^s \times c \times b^{q}$ - $b=2$ - 64 bits for full representation - 52+1 bits for mantissa or coefficient or fraction (about 17 decimal digits) - 11 bits for exponent ($\pm 308$) ![IEEE 754 internal binary representation. Source Wikipedia](./pics/fig_float_ieee754.png){width="80%"} ## Overflow and underflow errors - maximum $\pm 1.7976931348623157 \times 10^{308}$\ (provided by `sys.float_info.max` in Python) - `\alert{overflow error}`{=tex} - $(1.7976931348623157 \times 10^{308}) \times 10 = \texttt{inf}$ - smallest non zero $\pm 2.2250738585072014 \times 10^{-308}$ (provided by `sys.float_info.min` in Python) - it is also calculated as $2^{-1022}$ - you can work with a smaller number via so called [subnormal numbers](https://en.wikipedia.org/wiki/Subnormal_number) $2^{-1074} \approx 5 \times 10^{-324}$ - it might be slower - `\alert{underflow problem}`{=tex} - $(2^{-1074}) / 2$ produces $0$ ## Round off errors - `\alert{round off or truncation error}`{=tex} - $1.79769313486231\mathbf{ 6} + 20 \to 21.79769313486231\mathbf{8}$ - $1.79769313486231\mathbf{6} + 100 \to 101.79769313486231\mathbf{\_}$ - $1.797693134862\mathbf{316} + 1000 \to 1001.7976931348624\mathbf{\_\_\_}$ - how to mitigate - try to use numbers of the similar magnitude - do not trust the least significant digits