References:
names of PLs
(IMPERATIVE) C#, C, Swift, Java, Ada, Rust, Pascal, scala, Algol, Go, Obj-C, Fortan, C++, B, PL/I, X,
(SCRIPTING) Kotlin, Python, Julia, Javascript, PHP, R, Ruby, small talk,
(FUNC) Lisp, Scheme, ML, Haskell, Erlang,
(LOGIC) XLST, Prolog, SQL,
Assembler
Why so many?
What is a PL for ? - means of expressive algorithms. - tell computer hw do what you want. - way of organizing thoughts: func vs imperative. - ——- awaying ideas to others. - Knuth: Programming: art of telling another person what you want the computer to do.
To verify: Bubble sort n^2; insert sort n;
Insertion sort:
imperative:
insertion_sort (A)
for i in len(A) -2 down to 0
v = A[i]
for j in i+1 to len(A) -1
// A[j..len(A) - 1] is sorted
if A[j] > v break
A[j-1] = a[j]
A[j-1] = v
functional lang:
sort(A):
if len(A) < 2 return A;
else
let v be A[0] + R be A[1..]
return insert (v, sort (R))
where insert (v, S):
if len(S) == 0 return {v}
else
let w be S[0] + T be S[1..]
if v <w return v, S
else return w.insert(v,T)
Read book before class
Initial assignment
Get book.
Read chapter 1.
CSUG account.
Go to blackboard to take T0.
Checkout A(1) on the web.
Compiler:
Interpreter:
Java – bit code — can be interpreted (early state); or be JIT compiled to binary executable (modern ways).
X86 processor: interpreter inside. for slow instructions. old instructions?
Front end. Interpreter as an example: AST trees + symbol table ~= intepreters. Walk through AST and execute it with symbol table storing the data.
Compiler backend:
Compiler “Middle end”: 90% of a compiler. Optimization.
Bootstrap: write compiler of C using C lang; originally used by Pascal, x86.
Output of langs:
Java language -> web pages -> java byte code, small size. WebAssembly.
Latex -> pdf. VLSI -> circuits.
Compiler vs Interpreter.
What is hard about compilers?
Dynamic typing, even if strong typing. (strongly typed =~ type safe)
Phases of Compilation:
Front END: (program chars) -> scanner -> (tokens) -> parsing -> (parse tree) -> semantic analysis (static) -> (abstract syntax tree, or syntax tree) -> intermediate language generation -> (Intermediate form, IF) Middle: (IF) -> Optimization (IF) Back END: (IF) -> target code generation -> (**tive lang) -> native speficfic opt. -> (native code).
Theory: Regular experssion, context-free grammar, and everything else.
Regular expression is generator of reg language; Scanner is recognizer for reg language (valid tokens).
CFG is generator of CFL; Parser is recongnizer.
Parser vs semantic analyzer.
Syntax: grammar in context-free languages. Linguist: John Roundski? language hierachy.
Semantic analysis: everything else?
SLR(1) grammar, not LL(1) grammar.
Scanning.
Turning Machine Automata
Scanners to recognize tokens by regular expressions.
Stephen Kleene; U of Wisc; Mathcn; Recursion theory;
Kleene star (Kleene closure).
Church-Turning Theory: computability.
Scanner generators:
Production: LHS -> RHS
‘Derivation’: proof of syntactic correctness.
right-most derivation = canonical derivation.
Can parse anything in O(n^3) time; need to parse in O(n) time:
both LR/LL are PDA. push down automata; forest and trees; shift in tokens, build tree, and combine trees ;
SLR; SALR; (full) LR
Recursive decent vs. Table-driven topdown parsing:
Recursive decent. handwritten; ANTLR;
Table-driven. FMQ.
Program parsing:
Syntax tree: associativity and precedence.
1 program -> stmt_list $$
2 stmt_list -> stmt_list stmt | E
3 stmt -> ID := expr | READ ID | WRITE expr
4 expr -> term | expr add_op term
5 term -> factor | term mult_op factor
6 factor -> ( expr ) | ID | LITERAL
7 add_op -> + | -
8 mult_op -> * | /
EPS (a) = {true, false}
FIRST(a) = { t: t can start a };
FOLLOW(A) = { t: t can follow A in some sent. form };
PREDICT (A->a) = { t: t $\in$ FIRST(a) , or t $\in$ FOLLOW(A) if EPS(A)==true, or EPS if EPS(A) = false }
Wirths algorithm
for syntax error recovery:
Context-aware ‘smart’ paser: context-specific follow set: local follow set in addition to global follow set.
Recursive descent
stack of table-driven top-down parser contains the expected future program. ‘stack contains the future’
table:
stack: (top on left) P ; -> SL $$; -> S SL $$; -> read id SL $$; -> SL $$; -> S SL $$; -> read id SL $$; -> S SL $$; -> id := E SL $$
input: read A read B sum:= …
Semantic analysis:
attribute grammars (AG): labels on nodes;
relationships between nodes;
AGs work on trees – like parse trees or syntax trees. They assume that some nodes start out with interesting annotations: e.g., textual values of tokens for identifiers, strings, and numbers; built-in names and other “environmental” information in a table attached to the root (program) node. The AG then says how the annotations on other nodes depend on those of their children and/or parents. In a well-formed grammar, if you apply all the rules, you can decorate the entire tree.
EG: arch info on root node
EG: use attribute to describe how parser will build parse tree while parsing.
EG: use attribute to describe
AST_node expr()
case input_token of
id, literal, (:
T := term()
return term_tail(T)
else: error
AST_node term_tail(T1)
case input_token of
+,-:
op:=addop()
T2:= term()
N := new node(op, T1, T2)
return term_tail(N)
), id, read, write, $$:
return T1
else error
EG: bottom-up grammar: Below grammar is bottom up. All attributes are “synthesized” – they depend only on things below them in the tree.
It annotates the root of the parse tree with the value of the overall expression.
E => E + T E1.n = new bin_op(+, E2.n, T.n) // .n -> .node
E => E - T E1.n = new bin_op(-, E2.n, T.n)
E => T E.n = T.n
T => T * F T1.n = new bin_op(*, T2.n, F.n)
T => T / F T1.n = new bin_op(/, T2.n, F.n)
T => F T.n = F.n
F => - F F1.n = new un_op(-, F2.n)
F => (E) F.n = E.n
F => const F.n = new num(const.val)
F => id F.n = new ident(id.name)
<< show how this handles, for example, (a + 1) * b >>
Action Routines (AR)
If you can decorate an entire parse tree in one pass while parsing (that’s a big “if”), then the AG can be written in the form of ACTION ROUTINES – code fragments embedded in the RHSs of productions, with references to the “attributes” of the symbols in the current production. In this case you can actually get away with not building the parse tree explicitly – you just deal with the local neighborhood while parsing.
In principle, you can do all semantic analysis by decorating the parse tree (maybe while parsing; more likely in a separate pass [which would require that you actually build the parse tree]). It’s typically a lot easier, though, to write an AG for the parse tree that serves only to build an AST – to tag the root with a pointer to an AST. Construction of the AST can almost always be done with ARs (during parsing), allowing us to skip actual construction of a parse tree. Then we can use another AG to decorate the AST.
Attribute rules:
EG: attribute grammars on parse tree, to generate AST finally.
.n and .st are attributes. Uses n again, but also st, which is “inherited” – doesn’t depend only on things below. This st attribute serves the role of the left operand passed into term_tail in the example above.
E => T TT E.n = TT.n
TT.st = T.n
TT1 => + T TT2 TT1.n = TT2.n
TT2.st = new bin_op(+, TT1.st, T.n)
TT => - T TT TT1.n = TT2.n
TT2.st = new bin_op(-, TT1.st, T.n)
TT => TT.n = TT.st
T => F FT T.n = FT.n
FT.st = F.n
FT => * F FT FT1.n = FT2.n
FT2.st = new bin_op(*, FT1.st, F.n)
FT => / F FT FT1.n = FT2.n
FT2.st = new bin_op(/, FT1.st, F.n)
FT => FT.n = FT.st
F => - F F1.n = new un_op(-, F2.n)
F => ( E ) F.n = E.n
F => const F.n = new num(const.val)
F => id F.n = new ident(id.name)
Semantic analysis.
Type checking.
Dynamic type checking. Java array bounds. a = B[f(x)]
Runtime optmization. can only do when it can prove it is safe.
Definite Assignment: (Java, C#): machemism to check uninit vars. If anyone of paths has potential (does not reason about actual condition values) to use an uninit var, then compile would fail.
Sanity check on byte code. make sure come from authorized compilers.
Rules for annotating a tree, e.g. the AST.
EG: Typing rules:
a := b + c
Rule1: :={stat} = if L{type} = R{type} then OK else error;
Rule2: +{t} = if L{type} == int and R{type} == int then int; else if Lt,Rt \in {int, float} then fp; else error.
Attribute flow:
Bottom up: S-attributed: synthesized attribute only.
Top-down & Left-right: DFS L-R while parsing. L-attributed. More general:
Given a AST, and attribute rules.
Attribute rules: define the labels/relationships/attributes-propagations of nodes/neiboughhood.
Formal verified compiler. Attribute grammars. DoD set. hundreds of code. Automatically generated. grammars.
Float int cast: user-kernel cast?
P -> I
int dec: I
real dec: I
read: I
write: I
assign: I -> id E I
'+': E -> E E
Enforce rules in local neighbourhoods of a parse or syntax tree.
EG: A + B:
Dont need a parse tree if you have other place to store atributes of active productions
Action routines.
lambda calculus 1930s Church
Lisp ~1960 McCarthy
ML Milner - OCaml, by INRIA of French - SML, - Miranda, – Haskell.
Recursion instead of loops.
gcd(a,b) = a, if a=b gcd(a-b, a), if a>b gcd(a, b-a), if b>a
-- Euclid algorithm.
let rec gcd a b =
if a = b then a
else if a > b
then gcd (a-b) b
else gcd a (b-a)
if-then-else
: whole expression has a value. Everyone have an else
; or return assumed to return nothing on else if no else
in some language.
Tail recursive call: Tail recursion equal to loop iterations. Return value at tail.
But not all recursion can be converted to loop iteration.
Assembly implementation of tail recursion is a loop:
gcd(a, b) {
top:
if a == b return a
elsif a > b
a := a - b
goto top
else
b := b - a
goto top
}
Make a non-tail recursive call into a tail recursive.
continuation passing style: avoid all returns. compiler need to determine when to pass the function body.
let rec sum1 f low high =
if low = high then f low
else (f low) + (sum1 f (low + 1) high);;
// changed to
let rec sum2 f low high st =
if low = high then st + (f low)
else sum2 f (low + 1) high (st + (f low));;
To get rid of the extra parameter st
, defining a helper func:
let sum3 f low high =
let rec helper low st =
let new_st = st + (f low) in
if low = high then new_st
else helper (low + 1) new_st in
helper low 0;;
let a = 3 and b = 5 in expr
.
EG:
let rec f = .. g ..
and rec g = .. f ..
in expr
lambda exporession: first class functions: can pass, can return,
Fibonacci Num:
let rec fib n =
match n with
| 0 -> 1
| 1 -> 1
| _ -> fib (n-1) + fib(n-2)
// tail rec version:
let fib2 n =
let rec helper f1 f2 i =
if i = n then f2
else helper f2 (f1 + f2) (i + 1) in
helper 0 1 0;;
A constant time fib(n) ‘algorithm’:
F(n) = nearest whole number to ($\phi ^ n$)/(\squa {5}), where $\phi = (1 + \squa{5})/(2)$
Requires
Lisp:
ML:
EG: polymorphism: the type of function len
is polymorphism:
let rec len l =
match l with
| [] -> 0
| [_] -> 1
| hd::tl -> 1 + len tl;;
> _: len = `a list -> int <func>
REPL
#load “str.cma”
3+4 -: 7: int
#use “prog.ml”
< > <= >= work on everything except functions.
Ocaml: * comment *, inherited from Pascal.
$\lambda$ notation: let f = func a1 a2 a3 -> …
match expr with | var -> | var2 -> | _ ->
let primes5 = [| 2; 3; 5; 7; 11 |];;
primes5.(2) => 5
primes5.(2) <- 1234 * assignment *
g = {};;
type ‘a bin_tree = | Empty | Node of ‘a * ‘a bin_tree * ‘a bin_tree;;
type ‘a option = | None | Some ‘a
try expr1 with Foo -> expr2
map (fun x-> x*x)[2;3;5;7] => [4;9;25;49]
map: (a ->
a) -> a list ->
a list
compose f g x = f (g x);;
let h = compose f g;; // Currying: apply partial parameter of functions and later apply of rest parameters. (compose hd tl) [1;2;3] = hd (tl [1;2;3]) = 2
(*) 2 3 => 6
fold_left (*) 1 [2;3;4;5;7]
let rec fold_left f i l = match l with | [] -> i | h::t -> fold_left f (f i h) t;;
Currying: if func has 3 parameters, then only 2 are given; a new function will be returned which takes only 1 parameter, which is the third parameter in original function.
Powerful exmples:
let total_all = map total;;
total_all [[1;2;3;4;5]; [2;4;6;7;20]; [4;5;3;6;0] ];;
==> [15;]
unlimited extent let plusn n = func k -> n + k;;
n has unlimited extent
Currying: ( + ) : int -> int -> int
int -> (int -> int): given one int, it returns a function that is (int -> int)
let f x y = if y then x else 0;
f (g a) b
if g a
is very expensive and b is false: passing function around without evaluating the function.
Normal-order evaluation: passing unevaluated. Haskell. non-strict language.
applicative-order evaluation: passing evaluated. Ocaml, Java, C/C++; strict language.
Church-Rosser thereom: if application-order work, then normal order will work;
infinite
list: lazy evaluation, an optimized version of normal-order evaluation.type a stream =
Cons of
a * (unit -> `a stream);;
let hd: a stream ->
a =
function
| Cons(h,_) -> h;;
let tail: a stream ->
a stream =
function
| Cons(_, t) -> t();;
Lazy evaluation can incurr ‘wrong’ or different result compared with non-lazy evaluation: Imperative side-effects in a Functional style language: lazy computed once and store it for use in future, but imperative statement could change original compuation parameters and invalide the previous results. (Haskell has no such side effects); Ocaml allows lazy by lazy library.
Scope of a binding is a region of the program in which the binding is active.
Region can be:
textual region: static scoping – most lang; (different concept with typing).
temporal region: dynamic scoping – early Lisp, tcl, shell script language (what variable depends on where and who calls it, like bash env variable)
Scope:
func() { int a; int b; // b scope starts in C; but starts block start in C# }
Binding: name vs the things it names.
creation/destroy of object/bindings. deactive/reactive of bindings.
elaboration of declaration.
reference environment of an stmt or expr = set of active bindings.
Passing function A to another function B: whose data, or whose environment we want to use.
Closest enclosing scope.
Declaration vs Definition
.h files: declarations;
Three main categries of data: static, stack, heap.
Compiler implements them.
Static: lifetime = whole program; e.g.
Stack: lifetime = single function; e.g.
Stack figure:
----------------------------------------
|| || | | ||
|| || | | ||
----------------------------------------
frame, as known as ‘activation record’
frame pointer, compiler use this to find contents in the frame (e.g. arguments)
bookkeeping info: e.g old frame pointers; return address;
stack pointer, top of stack. last used words or first unused words. - Used for compute arguments address in a frame. - Because size of locals ad temps can be unknown at compile time.
calling sequence: Usually determined by ISA vendors, including HW features & conventions.
E.g push
instructions in ISA: change stack pointer + do stores.
E.g caller and callee saved registers.
E.g. stack conventions: if compilers obey the convention: different code generated from different languages & different compilers can call each other.
Two reasons why static allocated stack frame size is not good:
Heap: ‘new’-ed stuff (or implicit)
Nested subroutines, access local variables of another function’s frame. Offset might not be known at compile time(e.g. due to recursion of some routine between);
Solution: static links: every frame contain a stack link to its enclosing routine’s frame.
Subroutines: OCaml vs C vs C++
Nested scope:
Compiler sometimes avoid the dynimcally free of stack variables to save time; and reuse them for future nested scope that mutually exists, i.e. will never exists at same time. For example the a b in the following:
void foo(){
int x;
{
int a;
}
{
int b;
}
}
frame:
---------
a,b
x
---------
deep binding
when pass subroutine as an argument, need to remember the argument value when it is created.
let foo l k =
let rec helper l f i b =
match l with
| [] -> raise (Failure "list too short")
| h :: t ->
if i = k then f h
else if (b && h < 0) then helper t (( + ) h) (i + 1) false
else helper t f (i + 1) b in
helper l (fun x -> x) 0 true;;
This captures the "right" h in foo [1; -3; 2; -4; 5] 4;;
------------
f -----> ? h
helper
...
helper
helper
foo
------------
subroutines:
1st class: pass and return;
2nd class: can pass cannot return;
3rd class: no pass no return;
To implement 1st class funcs: “unlimited extent”
let g p = p 2;
let d() =
let x = 3 in ( + ) x;;
g (d());; ==> 5
// where to find x?
Today: Dynamic Scope; more on binding rules; lambda expressions; overloading, aliases, etc; Intro to scripting.
int a
proc first:
a = 1
proc second:
int a
first()
a := 2; second(); write(a)
static scope: 1 (use) dynamic scope: 2 (use latest recent name on the stack/association list/)
Today, most in static scope; but dynamic in few langs:
nested procedures.
deep-binding: whenever passed, capture the reference environemt;
shallow binding:
deep/shallow binding can be combined freely with static/dynamic scopes.
subrtin closure: (code + ref env)
object closures: a static/manual way to pass a subroutine around instead of using nested subroutines.
subroutines: bar{ int x; foo{ y = x // this x is said to capture the x from the surrounding context, declared in bar; } }
What is good to use lambda?
“capture things” in lambda expressions:
fun x -> x * y // y is captured from surrounding context.
def plus x = fun y -> x + y;;
def plus3 = plusx 3;; // subroutine closure = |plusx | 3 |;
// plus3 5;;
// ==> 8
lambda in imperative language: ruby, scala, C#, Java, C++ (Java and C++ does not have unlimited extent).
In C++, capture list when define a lambda expression.
auto plusx (int x) {
return [x](int y){return x + y;}; // lambda expression, a function with no name; a body of func; a capture list [x]
//return [&x](int y){return x + y;}; // &x here is not safe.
}
//...
auto plus3 = plusx(3);
count << plus3(5); // ==> 8
C++: capture list
More than one name for the same thing.
int x;
void foo(int &y){
count << x << y << x + y ;
}
foo (&x);
The opposite of aliases: Overloading
The same name for more than one thing. (not exactly the same as polymorphism.)
int a = square(3) float b = square(3.14)
Things (function or object) that works with multiple types.
Sometimes: Overloading is refered as “ad hoc polymorphism”; ad hoc means manually created several things then you give them the same name; can be seen as a poor variant of polymorphism.
An implict parametric polymorphic function:
let len l = match l with
| [] -> 0
| h::t -> 1 + len t;;
len: `a list -> int
`a is called type parameter, which is a hidden parameter to the function.
subtype polymorphism: class t of its derived classes;
Info flow: ‘Less than’ intrinsics for user to use?
parametric:
Let S = .....
set<string> S
.Motivations:
Dynamic typing.
Midterm review.
Chapter 1-4, 11, 13
The most distinguish feature: Dynamic typing;
write code without declaring variables; good for small programs;
Static scoping: looking outward for declarations; but scripting does not have declarations!
Perl: global unless declared otherwise; $foo = 1;
is global by default; local foo; my foo2;
are local variable; local foo
is using dynamic scope; my foo
is using static scope; our foo
, the same;
LLM: dynamic/static typing vs dynamic/static scoping.
PHP: local unless imported;
Pyhton: local if written; (if you assign to any variable, this variable will be local; if you only read it, its global ??? )
Ruby: foo
is local; $foo
is global; @foo
is instance of a class; @@foo
is a class;
R: foo <– 3 ; foo <<– 3;
regular expression.
grep/sed/awk/egrep: extended regular expressions: quantifiers:
* + ;
exact three instances: {3};
2 to 5 instance: {2,5} ; {3,};
^ for begin; $ for end of line;
\;
advanced REs: $foo ~=s/ab/cd/g
grab pieces of string.
$_ default syntax for matches in perl;
$_ = "-3.14e5"
if (/^[+-]?((\d+)\.|(\d*))\.(\d+))(e([+-]?\d+))?$){
1 2 3 4 5 6 7
}
// ? means optional; [] one char; \d digits;
// use $1, $2, $3, ... for the matching parts
Statements: no value; executed for its side effects;
Expressions: has value;
f(a+b, g©, d(e,h), h)
In which order to evaluate? : first a+b, first h?, first g© ?
Compiler might reorder the evaluation order to reduce the register pressure.
Language manual does not have specifications on the order; and compiler want freedom to reorder the evaluation order;
==> therefore, do not write code where g© can influence the behavior of d(e,h); otherwise, it will result in to undefined behavior;
S = T() + TT();
Compiler can evaluate TT() first then T() as optimizations;
a + b + c Associativity is considered unsafe (to reorder): overflow/underflow can zero some operands (b is extra large floating point and c is extra small float point, then c can be zero out with no effects to final addition result);
One ad-hoc implementation case of Lazy evaluation’
Q: reorder the multiple or conditions: c1 | c2 | c3 | c4 | c5: which one should be evaluated first in order to make faster decision.
Seqencing
Selection
Iteration
Subroutines
Non-determisim: order does not matter;
Concurrency
if
case/switch
if c1
then if c2
then
else ... (which if is this else belongs to)
// solution
- match the closest ones;
- use terminators: end
case/switch:
fast implementaiton: a dictionary match constance (address) to lambda expressioni (code)
A dictionary abstract data type; data structure are implementations; which data structures should we use to implement case/switch?
If no match
For cases with single value:
// bad decision to have break in switch stmts in C;
// most other language don't have notion of fall through
switch (x) {
3:
5:
7:
9:
}
// characteristic array: array[0, max-value], each element is empty (if no such case) or stores a pointer (code pointer for that case)
// hash table: compiler knows all the possible hash values --> can pick (a perfect) hash func with no collisions.
// linear search array: arrary of <case, code-ptr>
For cases with ranges:
case expr of
1 .. 48:
97 .. 283:
900 .. 1024:
1234 .. 2302:
esac
// decision tree O(log(n)), no insert/deletion, can do redistribution if not even tree;
// or binary search O(log(n)),
While, repeat .. until, do .. while
Ada: bread to the second/third innerest loop. break B
;
top:
r1:=condition // >=1 inst
if !r1 goto continue
<body>
goto top
continue:
// improve 01
r1:=condition
if r1 goto continue
top:
<body>
r1:=condition
if r1 goto top
skip:
// improve 02
goto test
top:
<body>
test:
r1:=condition
if !r1 goto top
for (i = first; i<= last; i++){
// body
}
==>
i = first
while (i <= last){
// body
++i;
}
// but got complicated in newer C, where for (int i=first;...), the i is local to body.
Built in iterators for the language, like
True iterators: Clue, Python, C#
Iterator objects: Java~, C++~,
lambdas: Ocaml, Ruby, Lisp, ML
open Printf;;
let show n = printf "%d\n" n;;
let upto lo hi =
fun f -> let rec helper i =
if i > hi then ()
else (f i ; helper (i + 1)) in
helper lo;;
upto 1 10 show;; =>
1
2
3
4
5
6
7
8
9
10
- : unit = ()
sum = 0 # => 0
[ 1, 2, 3 ].each { |i| sum += i } # => [1, 2, 3] # array itself
sum # => 6
# or
sum = 0
for i in [1, 2, 3] do # 'do' is optional
sum += i
end
sum
Python, Clue, C#
def uptoby(lo, hi, step):
while True:
if (step > 0 and lo > hi) or (step < 0 and lo < hi): return
yield lo
lo += step # ignore overflow
# call it
for i in uptoby(1,10,2):
# body
Clue put the iterator’s stack just at the top of caller’s function; if other function called in the body of loop, then the stack frame would just be put on top of the iterator’s stack frame; However, this violates the calling convention defined by the ISA vendor that is used by other languages.
condition can be hard to figure out whether <= or >= if the sign of step
is not known statically.
if c-- > 0 goto top.
Today:
iterator objects;
type system.
in Java:
List<foo> myList= ...;
for (foo o: mylist){
// use o
}
==> a syntax sugar of underlying:
for (Iterator<foo> i = myList.iterator(); i.hasNext()){
foo o = i.next();
// use o
}
==> for a iterator, must have three functions: iterator(), hasNext(), next();
(LLM: will we have buffer overflows in these data structures, i.e. iterators? Or can buffer overflows elsewhere can change the behavior of iterators? How?)
in C++:
list<foo> my_list;
...
for (list<foo>:: const_iterator i = my_list, begin(); i != my_list.end(); i++){
// use i
}
// iterator =~ (smart) pointer
// .end() is a pointer that has just passed the last element.
in C#
foreach(foo o in myList){
// use o
}
==>
for (IEnumerator<foo>i = myList.GetEnumerator(); i.moveNext();){
foo o = i.Current; // accessor
// use o
}
// The interface implemented as
class List : IEnumerable {
...
public IEnumerator GetEnumerator() {
node n = head;
while (n != null) {
yield return n.content;
n = n.next;
} // NB: no return stmt
}
}
// ==> the compiler define IEnumerator class as a special calss; in facing of this and `yield return` statement inside; the compiler will automatically create
// - the new version of .GetEnumerator()
// - the .MoveNext();
// - the .Current
//
// iterator stored on heap? implicit iterator object.
What is a type?
What are types for?
Assembly does not have types. BCPL (before C), untyped. Can do anything on any bit streams.
Type check:
type of ‘Prime’: assigned a value, make sure it is a prime number; ==> can be expensive.
Contain three things:
Ada: does not have Coercion /kouezshen/; safe is more important.
LLM: capkernel
different types in capkernel does not have expression
. a mixed op of differnt types.
data of different types in an expression? what we should do in capkernel?
code of differnt types in an expression? what we should do in capkernel?
code and data in differnt types in an expression: error;
ccall –> a cast from one type to another type? No. code and data changed to another set;
if x then e1 else e2
e1: 'a * int
e2: char list * b
==> expression inferred as <char, int>
Unification in aritificial intelligence.
Proglog: unification.
C++: template. Turing complete by statically compiling: can do any computation.
e.g. Prime<3> x.
Structural equivalence:
Name equivalence:
Type Systems
Polymorphism
Structural vs. Name equivalence
e.g.
struct{ age: integer name: string } college; person;
structural equ: college type is the same as person type; but name equ does not.
type temp = integer; // alias type; whether it introduce new type depends on the language. Yes or no or your choice (in Ada; type temp = new integer;)
var A: temp;
var B: integer;
conversion/coercion/cast
conversion: safe
coercion: safe, implicit converstion
cast: language impl. dependent; bit there, and pretend as a certain type.
Union { t1 a; t2 b; }
d = (double)n
static_cast
dynamic_cast
reinterpret_cast
const_cast
Subtype: parent code works for child.
Parametric: type of parameter can be viable.
length(<list of type x>)
, implicitin C++, explicit: class list <T> { }
;
template <class T>
class stack {
T[100] contents;
int tos = 0; // first unused location
public:
T pop();
void push(T);
...
}
...
stack<double> S;
can also be:
class stack {
void*[100] contents;
int tos = 0;
public:
void* pop();
void push(void *);
...
}
but will lose type checking (use cast all over the place, not type safe in C++)
C#: reification.
stack
stack
stack
in C# struct and class are different; struct on stack; class objects on heap;
Java: erasure
in Java: struct and class objects/instance are all on the heap.
JVM does not have to know whether you are using generics or not; (avoid JVM changes when introducing generics (since v5));
Java compiler will give JVM the void * version, with all static checked safe properties, and also all run-time checked safe properties shipped with the bitcode.
in both C# and Java: compiler type-checked generic.
C++: no type check in generic; but elobration-time instantiation; very powerful. (turing complete template). but ugly error message (content timing); C++20 “concepts”
When do instantiation, like stackPerson
into the code and compiler it; report erros if compilation fail.
Prime<13> n;
C++20 “concepts”
Java/C# use Interfaces to check generic types:
public static <T implements Comparable<T>> void sort(T A[]) {
...
if (A[i].compareTo(A[j]) >= 0) ...
...
}
...
Integer[] myArray = new Integer[50];
...
sort(myArray);
interface Comparator<T> {
public Boolean ordered(T a, T b);
}
class Sorter<T> {
Comparator<T> comp;
public Sorter(Comparator<T> c) { // constructor
comp = c;
}
public void sort(T A[]) {
...
if (comp.ordered(A[i], A[j])) ...
...
}
}
class IntComp implements Comparator<Integer> {
public Boolean ordered(Integer a, Integer b) {
return a < b;
}
}
Sorter<Integer> s = new Sorter<Integer>(new IntComp());
s.sort(myArray);
Heat and power for single core processors.
Human brain’s efficiency: pattern matching.
Multicores (since 2004).
More and more specialization. GPU, compression/decompression engine (chips), encryption/decryption, multi-media engine (radio signals).
Amdahl’s Law: GENE. is a law. (csci 258). limit of the making your program faster. optimization part.
Processor vs Cores
Processor: one socket.
Core: one core with multiple HW threads – “hyperthread”
runtime app: program with language threads. More language threads than the kernel threads serving this program. – Scheduler 01 (this class)
OS kernel: kernel threads, process (~= address space, multiple threads). More kernel threads than hardware hyperthreads. – Scheduler 02 (256 OS class)
HW: hyper-threads.
Two programming models for communications between threads.
shared memory tends to be faster than message passing.
Problem with shared memory:
int x = 0;
T1: x++; // in asm: r1 = x; r1++ ; x = r1
T2: x++; // in asm: r1 = x; r1++ ; x = r1
x == ?
Synchronization: exclude bad interleavings.
Four-way world of mechanism
atomic condition
busy-wait keep asking
sched sleep/wakeup, such as interrupts.
A scheduler keeping track of all threads’ state (which is ready to run).
A scheduler usually implemented with busy-wait sync; and it can enable application to use either busy-wait or scheduler based sync.
Thread creation
Thread implementation
Data-race free programming
Spin locks
Thread creation:
static set
co-begin: Algol 68, Occam, SR
* parallel loops
lanuch-on-elaboration: Ada, SR
fork(join): Ada, Modula-3, Java, C#, OpenMP.
implicit receipt: DP, Lynx, RPC systems. (like a func call, but sends to server for execute)
early reply: SR, Lynx. Cf. separated new() and start() in Java.
A thread in Java ******************************(LLM: a trampoline code? a new calling convention? )
E.g Java: extends Thread: you implement td.run()
, you call td.start()/join()
; td.start()
will do everything necessary to fork/finish a thread.
=> Early Return:
// define it
f(){
...
reply();
...
}
// call it
f();
Lanuages: Java, C#, Ada, Go
OpenMP: parallel loops
libraries: pthreads, winthreads (on Windows)
One researcher? problems if parallel in a lib but not in language.
data races: bad.
sync races: may be good.
“ordinary data” vs. synch data.
Processor is designed to provide an ‘illusion’ that the program/thread are executed at once.
initialization example
// ready == false
p = new foo(args)
ready = true while (!ready) {}
// use *p
x86: store always in order;
ARM: store can out-of-order. ************************
One way to solve: Java: volatile (C/C++ atomic): hardware will not do ooo stores.
Another way: Sync block.
==> Data-race free code
Faster than scheduler based locks.
SL needs something else to be atomic.
Atomic load/store.
SL:
RMW: read-modify-write; fetch t $\phi$
test-and-set; compare and swap; or load-linked / store-conditional (ARM);
swap; fetch-and-add.
acquire: while (tas(L)); // test and set. out of loop until change from false to true.
release: L = false;
Problem: contention.
memory
----------------------------
cash cash cash cash cash (privilage cache; a write will kick out any other's cache)
p p p p p
Variant: test and test and lock:
Voluntary yields.
Every thread and its coroutines, has its own stack.
A ‘transfer’ function: exe from one stack to another.
Transfer function (a context switch between coroutines):
return;
// single processor
reschedule:
t : cb := dequeue(ready_list)
transfer(t)
yield:
enqueue(ready_list, current)
reschedule
sleep_on(q)
enqueue(q, current)
reschedule
LLM: === Automatically split sequential program into multi-threads using program analysis??
not yield by itself, but force one thread to yield. Use timer interrupts (in OS) or signals (in library package) to trigger involuntary yields.
Requires that we protect the scheduler data structures:
yield:
disable_signals()
enqueue(ready_list, current)
reschedule
re-enable_signals()
add lock
Disabling signals doesn’t suffice:
yield:
disable_signals()
acquire(scheduler_lock) // spin lock
enqueue(ready_list, current)
reschedule
release(scheduler_lock)
re-enable_signals()
disable_signals()
acquire(scheduler_lock) // spin lock
if not <desired condition>
sleep_on <condition queue>
release(scheduler_lock)
re-enable_signals()
Scheduler-based synch
Java concurrency
Bounded buffer
Spin lock in C++/C: data race in the lock when doing test and set: must make the operation atomic.
A special counter: two methods: P, V. originated from two dutch words.
Example impl:
type semaphore = record
N : integer -- initialized to something non-negative, the counter.
Q : queue of threads
procedure P(ref S : semaphore) : // reduce counter
disable_signals()
acquire(scheduler_lock)
if S.N > 0
S.N -:= 1
else
sleep_on(S.Q) -- give processor to others
release(scheduler_lock)
re-enable_signals()
procedure V(ref S : semaphore) : // add counter
disable_signals()
acquire(scheduler_lock)
if S.Q is nonempty
enqueue(ready_list, dequeue(S.Q)) // wake up one thread in the queue.
else
S.N +:= 1
release(scheduler_lock)
re-enable_signals()
shared buf : array [1..SIZE] of data
shared next_full, next_empty : integer := 1
shared mutex : semaphore := 1
shared empty_slots, full_slots : semaphore := SIZE, 0
procedure insert(d : data) :
P(empty_slots)
P(mutex)
buf[next_empty] := d
next_empty := next_empty mod SIZE + 1
V(mutex)
V(full_slots)
function remove returns data :
P(full_slots)
P(mutex)
d : data := buf[next_full]
next_full := next_full mod SIZE + 1
V(mutex)
V(empty_slots)
return d
1965 - semaphors 1970 - what could be better than semaphors: Monitors.
Combine the data and lock together; rather than in different places.
A monitor: Association between the lock and data; a class with its own data and lock.
Tony Hoare.
Entering queue;
Condition queues;
Urgent queue (most language does not use it):
if !cond wait(Q1) // being put in urgent queue
// another thread cond = T signal(Q) // wake up urgent queue and run immediately by giving up core by sleep().
// This can be inefficient: sleep() to give up cores is not necessary: can run on other cores; ==> do double checks
while !cond wait(Q1)
|
–Queue – | Monitor | –
// LLM: how the monitor implemented?
//
class BB {
final private int SIZE = 10;
private Object[] buf = new Object[SIZE];
private int nextEmpty = 0;
private int nextFull = 0;
private int fullSlots = 0;
synchronized public void insert(Object d) throws InterruptedException {
while (fullSlots == SIZE) {
wait();
}
buf[nextEmpty] = d;
nextEmpty = (nextEmpty + 1) % SIZE;
++fullSlots;
notifyAll(); // explain why!
}
synchronized public Object remove() throws InterruptedException {
while (fullSlots == 0) {
wait();
}
Object d = buf[nextFull];
nextFull = (nextFull + 1) % SIZE;
--fullSlots;
notifyAll(); // explain why!
return d;
}
}
BB solution in Java 2 without using notifyAll is quite a bit harder:
class BB {
final private int SIZE = 10;
private Object[] buf = new Object[SIZE];
private Object producerMutex = new Object();
// waited upon only by producers
// protects the following:
private int nextEmpty = 0;
private int emptySlots = SIZE;
private Object consumerMutex = new Object();
// waited upon only by consumers
// protects the following:
private int nextFull = 0;
private int fullSlots = 0;
public void insert(Object d) throws InterruptedException {
synchronized (producerMutex) {
while (emptySlots == 0) {
producerMutex.wait();
}
--emptySlots;
buf[nextEmpty] = d;
nextEmpty = (nextEmpty + 1) % SIZE;
}
synchronized (consumerMutex) {
++fullSlots;
consumerMutex.notify();
}
}
public Object remove() throws InterruptedException {
Object d;
synchronized (consumerMutex) {
while (fullSlots < 0) {
consumerMutex.wait();
}
--fullSlots;
d = buf[nextFull];
nextFull = (nextFull + 1) % SIZE;
}
synchronized (producerMutex) {
++emptySlots;
producerMutex.notify();
}
return d;
}
}
Solution using Java 5 locks is efficient and arguably more elegant:
Find a clean point that a thread can be kicked to run.
class BB {
final private int SIZE = 10;
private Object[] buf = new Object[SIZE];
private int nextEmpty = 0;
private int nextFull = 0;
private int fullSlots = 0;
Lock l = new ReentrantLock(); // new
final Condition emptySlot = l.newCondition(); // new
final Condition fullSlot = l.newCondition(); // new
public void insert(Object d) throws InterruptedException {
l.lock(); // new
try { // new
while (fullSlots == SIZE) {
emptySlot.await();
}
buf[nextEmpty] = d;
nextEmpty = (nextEmpty + 1) % SIZE;
++fullSlots;
fullSlot.signal();
} finally { // new
l.unlock(); // new
}
}
public Object remove() throws InterruptedException {
l.lock();
try {
while (fullSlots == 0) {
fullSlot.await();
}
Object d = buf[nextFull];
nextFull = (nextFull + 1) % SIZE;
--fullSlots;
emptySlot.signal();
return d;
} finally {
l.unlock();
}
}
}
One to one corresponding to machine types (Still valid in CHERI?)
Integers (8/16/32/64/):
Addresses: pointers (next class)
Floating-point numbers (IEEE 754 std)
Multibyte Characters:
Bigints:
Multiprecision Arithmetic:
bc
Some exotic Scalar Machine types: Decimal Numbers (binary coded decimal)
? - functions? - classes - objects - tensors
[] : 3[a] in C. –> C does not have array impl, but just treated as pointer arithmetics.
() : matlab
shall we track the size of arrays?
multi-dimensions
The layout of arrays:
address calculations:
return addr overwritten attack
malloc
C: ASCII vs Unicode. UTF-8: 0-127 use a single byte; 128+ use multiple bytes (up to 4).
Java: change Unicode representation to fix bytes length.
layout rearranged
All fields occupy same space
Memeory management
address space. Address, Pointers (a language notion of addresses), Reference
Recursive Type: e.g trees.
An exercise in Pointers: int i[10], *pi, **ppi; ** (ppi + 1) ==> seg fault, * (* ppi + 1) ==> value of second element.
Explit memory Management:
Garbage Collection.
Variants of GC for C/C++: cannot collect all garbage. LLM: CHERI can collect all for C; can we do similarly for C on legacy architectures???
in general we cannot know what is going to be used in the future since it depends on conditions
safe approximate: no reference to the object
need to distinguish pointers and integers
Impl. of GC:
Tracing collectors
generational: generations of data.
Stack | Heap
|
stooges ptrs | objects/blocks
|
next time: prologue, epilogue, calling conventions. parameter passing modes: call by value/reference/result/name; var num of arguments.
Regiter Window: e.g. 32 regs window in 128 physical regs.
language’s value model vs reference model of variables
pass by
Default Parameters;
Named Parameters: print(3, color=red) => compiler will pass all other parameters.
Variable # of parameters: printf;
some compilers will understand printf
: can check the types in the constant string.
if printf(str_var, ...)
: compiler cannot check for unknown variable strings.
int printf(...){
while(1){
switch(next % marker):
real: float f = next_arg(float);
int: int i = ... (int)
bool: ...
}
}
// IN Java, C#: variable parameters with known types.
nonlocal returns.
Goodenough
paper: a replacement model of exceptions. try/throw/catch.
try{
} catch (exception_type1 ex1){
} catch (exception_type2 ex2){
}
Three implementations:
C: (bad way): setjmp(), longjmp().
if (!setjmp(&jmpbuf)){ // long jump back here when exception, and setjmp will be pretended to return true.
// normal execution
// volatile int n;
// n++; // will write to memory; otherwise, longjmp will restore old reg values and this will be ignored.
// ---->--->---> longjmp(&jmpbuf) => has to know jumbuf to specify where it jump to.
}else{
// exception handler
}
External events.
Legacy way and modern way.
Modern: Create symatric handler threads during initialization. E.G. Graphic systems. Lambda expression as handler function.
class PauseListener implements ActionListener {
public void actionPerformed(ActionEvent e) {
// do whatever needs doing
}
}
...
JButton pauseButton = new JButton("pause");
pauseButton.addActionListener(new PauseListener());
// semantically equal to
// pauseButton.add(lambda(e){...})
Sequential handlers:
trampoline
main program
|
'-----------------------------------------> kernel
|
event signal <-------------'
handler trampoline rti
|
<----------'
|
'--------->,
|
,<------------------------'
|
// may cause deadlock (priority invertion: handler>main but main holds lock)
Object Orientation.
Simular: 1979? have all above features.
Object-based langs:
Javascript: create new object by saying “make something like that”
type extensions:
// Java
Parent p ;
Child c = new Child();
P = c;
p.f(); // which version do we get? dynamic binding: child methods.
However, in C++: Abstract type vs concrete type. ===> virtual methods.
// C++
Parent *p;
Child c;
p = &c;
p -> f(); // which version? virtual vs non-virtual ==> abstract vs concrete type.
Golbals, locals, heap, modules, objects, …
Public.
Private.
Protected. Slightly diff in C++ vs Java: Java visible to current package.
Header files:
Mangling: extra information to encode the types/methods. myclass$#adfdsafsagfhiuhoi
not found.
LLM: Is the heap removable? Heap is not included in the C semantics. why not have everything on stack? a dynamic sized stack at runtime? To remove the heap completely?
Garbage collection: stop the world and search the memory: can we use this ‘inspection’ capability to inspect the memory states?
std::mutex my_lock
{
std::lock_guard _(&my_lock);
break;
return;
thrown;
// can break out anywhere but the desctructor always be called thanks to my_lock;
}
super(a,b,c) in Java;
void Child(…): parent(…), member1(a), member2(…,member1,…){ // ^ copy constructor go called // ^ will still call empty constructor if empty.
member1 = a; // assignment operator got called;
}
==> Difference:
foo b; // initialization to b foo f = b; // assignment to f without initialization
foo f, b; // init to both b and f f = b; // assignment.
Parent vs Child methods, VTable for virtual methods
A struct with methods:
------- a vtable for each class ---> Vtable:
------- parent methods
------- parent methods
------- parent methods
------- child methods
------- child methods
------- child methods
------- child methods
VTable:
------- virtual method
------- virtual method
// Example in text book
Class widget{
interface sortable {
}
interface graphable{
}
}
Class named_widget extends widget implement sortable {
}
Class augmented_widget extends named_widget implements graphable {
}
// augmented_widget
--------- a pointer --> augmented/widget/vtable
---------
---------
---------
// sortable, at distance 'offset' from augmented_widget
--------- a pointer --> sortable
--------- --------- (=offset)
---------
---------
// graphable
--------- a pointer --> graphable
---------
---------
---------
C++ constructors and destructors.
Foo f; // f() //
class{
foo() = delete; // if you do not want default constructor, do this. compiler will check all declarations are not using default constructor.
}
foo f = g; // f(g); f g same type; copy constructor.??
gg = my_func(f);
fun(my_func(f));
void my_func(foo x){ //==> copy constructor is called when passing f by value.
foo t;
return t; //==> copy constructor can be called during return;
a,b,c = ...;
return foo(a,b,c); //==> this is an object somewhere; manual says this object should be created where this parent func is being called, to avoid a call to copy constructor; impl: a callback pointer to the caller i.e. in `gg=my_func(f)` or in `fun(my_func(f)) ; side effects.
}
f(foo(a,b),...)
f(y, ...)
foo: temporary constructor, pass it by copy constructor, destroy old one. ===> waste.
C++11: move
constructor.
Copy constructor: foo(const foo& other){}
Move constructor: foo (foo&& other){} // foo&&
is r-value reference. move constructor will copy the scalars and pointers to new object, and also nil the pointers in the old temporary object during ‘copying’. When destructor of the old temporary object get called, it will do nothing since already a nil, this will save time, since the object on the heap is not freed/re-created.
LLM: linear capabilities! cannot be copied; caller gives out
the capability
x = y;
// l/r value: l value has a location in memory; r value might not have location in memory;
p = &4; // 4 is not an l-value, but an r-value, and it does not have an address so invalid stmt;
E.G.
std::move<T>()
foo f = ....;
A.insert(f); // a copy constructor, f still valid after this line;
A.insert(move(f)); // a move constructor, f will be 'moved' and passed to A.insert; and f will not be valid after this line;
class widget{
interface sortable{
interface graphable{
class named_widget implements sortable{ // xyz(sortable s) { s.compare(T); }
class augmented_widget implements graphable{
aug/named widget view
+++++++++ --> vtable --- widget
--plain-- -------------
--------- -------------
---------
---------
+++++++++ --> vtable --- sortable for named_widget
- named - -- offset to named_widget, a
--------- -------------
---------
+++++++++ --> vtable --- graphable
- augm - -------------
--------- -------------
---------
named_widget w;
xyz(w); // passed in the sortable view of the object w; i.e. srtable = w + a; inside xyz, will get back to w by srtable - a;
xyz(sortable s){
s.compare(T);
// assembly translation:
r1 = this; // this of sortable;
r2 = *r1 ; // get the vtable;
r1 -= *r2 ; // get the this of named_widget; the concrete obj.this
r2 += 3 * wordsize ; // get the method of third
r2 = *r2;
call(r2)
}
class X: public P;
class Y: public P;
class Z: public X, public Y{
}
X
P Z
Y
=> P X Y Z or P Y X Z
or
P X
Z
P Y
allow old classes to be compatible.
defalut methods in interface vtable ?
Reference 1 Widely used parallel programming systems: Shared Memory Message Passing Distributed Computing Language Java, C# Extension OpenMP Remote Procedure Call Library pthreads, Win32 threads MPI Internet libraries Library augmentation for legacy languages still dominate today. While the library approach requires no changes to the syntax of languages that use it, compilers must refrain from performing optimizations that may introduce races into programs with multiple threads.
Reference: 09/08 Hello world from diff languages: Ada Haskell Ocaml Prolog Scheme … Interpreter: better error reports: have source code at runtime. 09/22 FIRST(each_rightHS); Follow(LHS); PREDICT(production of each rules) 11⁄10 C#/Python/Ruby: iterator permutations. def it(n) for i in 1 .. n yield i Stack push and pop. Hash table.
It is not yet clear to what extent, and in what problem domains, we can expect compilers to discover good algorithms for problems stated at a very high level of abstraction. In any domain in which the compiler cannot find a good algorithm, the programmer needs to be able to specify one explicitly. Q/A What distinguishes the front end of a compiler from the back end? What is the purpose of a compiler’s symbol table?
If you could revise
the fundmental principles of
computer system design
to improve security...
... what would you change?