Programming Assignment 6 Functions & File I/O 75 PointsDay Attendance Cost ($) 1 450 24.99 2 120 15.00 8 500 35.75 9 750 7.00 15 425 24.00 16 600 30.50 21 225 25.00 22 150 20.00 23 350 15.00 30 175 12.50 31 275 22.50▪ Write a program to read the dec_events.txt file and create a report. ▪ Store the report into a file named report.txt. ▪ The report should contain the following data for each event, printed in columns, with column headings:▪ Below the event table, print the total number of people attending events, the total revenue for the events and the average cost per person.▪ You should have separate functions to: o Read a line of data. o Calculate the revenue for an event and return it. o Print one line in the report. o Print the total attending, total revenue, and average cost per person. ▪ You can put the statements to accumulate the total attendance and total revenue right in main. ▪ Your program should work with files containing fewer or more records (lines) than those shown in the table above.*** A similar example program, labelled “Student Grades” is posted under Content for Unit 6.Code your program. Save it in a file named events_xxx.cpp where xxx are your initials.Compile, run and test your program.
Here you’ll use a variant of the deep deterministic policy gradient algorithm to learn to control a two-joint arm. The arm will have the same dynamics as in Assignment 1 except that the time step Δt will now be 0.1 s. Whereas my posted code file DDPG.m assumes that Δt = 1 (and therefore doesn’t mention Dt at all), you will introduce a Dt variable and write your code so it works with a Dt = 0.1. You’ll implement the state dynamics in a function called f which you’ll place at the bottom of your m-file. (To see how to define a function at the bottom of an m-file, look at DDPG.m). Your f function will take the state and action, s and a, as inputs, and put out the state at the next time step, s_next = f(s, a). Your f should handle single examples of s and a, not minibatches. It should also prevent the joints going outside their motion ranges, defined by q_min = [-2; 0] and q_max = [1.5; 2.5], as in Assignment 1. And it should Euler-update q before q_vel and q_acc. Also at the bottom of your m-file, please insert the function test_ policy from the sample code DDPG.m, but modify it so it calls your f function in the appropriate place. Use global variables to pass information other than their inputs to your f and test_policy functions, i.e. write 2 global n_q psi zeta q_min q_max n_steps Dt r in your m-file right after clear variables; and then, inside f and test_policy, list the global variables they need (much as I did for test_policy in DDPG.m). Define your reward function this way: r = @(s, a) -10*(s – s_star)’*(s – s_star) – 0.01*a’*a; where s_star = [0.5; 0.5; 0; 0; 0; 0]. When computing the return G, use no discount factor, or in other words set = 1. To make it easier for Ankit and me to compare your results with mine and with other students’, please seed the random number generator by writing rng(4) into your code right before you create your networks. Create all your networks using the posted function create_net.m. In this variant of DDPG, you will not use a Q_est network. Rather, you’ll decompose the action-value function Q into a combination of more-basic functions, in the hope that by exploiting its internal structure you can improve learning. This question will be an exercise in making multiple networks operate together, and particularly in figuring out what signals to backpropagate through which networks. The decomposition of Q will work like this. First, we define the value function V (s) = Q (s, (s)), i.e. V (s) is simply Q (s, a) when a is chosen to fit the policy , or in other words it’s the return we’ll get from now on if we’re currently in state s and we choose all our actions, including the current one, using policy . Next, we observe that Q (st, at) = r(st, at) Δt + V (st+Δt) = r(st, at) Δt + V ( f (st, at)), i.e. Q = V ⸰ f + r Δt. Your job is to code a variant of DDPG that has no Q net but instead works with networks V, f , r, and also target networks V + and f + (but no r + or Q + ). 3 Each of your six networks should have four layers and relu activation. The networks f , f + , V , and V + should have 100 neurons in each hidden layer, r should have 50, and should have 250. Initialize all your networks’ weights using (as the second input to create_net) the rescale vector [0; 0.5; 0.5; 0.5]. Create the networks immediately after the line rng(4), and in this order: , f , r, V , and then the target networks. If you structure and initialize your network correctly, then the first five columns of mu.W{end}, before learning begins, should equal -0.0338 0.0159 -0.0517 -0.0349 0.0364 0.0100 0.0020 -0.0657 0.0905 0.0163. You should train r in the obvious way based on minibatches drawn from the buffer, using a loss function that is half the square of the error e r = r(s_, a_) – r_ where s_, a_, and r_ are from the buffer. Not so obviously, you should train f using a loss function that is half the square of e f = V ( f (s_, a_)) – V (snext_), i.e. f needn’t necessarily predict the correct snext_, but it should predict a new state whose estimated value matches that of the true snext_ (because that’s easier than predicting snext_). You will have to work out how to train V — by finding an equation analogous to the Bellman equation but holding for V rather than Q, and then rearranging your equation to define an error signal analogous to the Bellman error but for V . You will also have to figure out how to adjust the policy network using the networks you have. Use the hyperparameters eta_mu = 3e-8, eta_f = 1e-4, eta_r = 1e-2, eta_V = 1e-4, tau = 3e-4, a_sd = 0.1. And for all learning, use the adam optimizer with its standard hyperparameters. Run 1000 rollouts in all, each with a duration of 2 s, i.e. 20 times Dt. Before learning begins, and then after every 100 rollouts, call test_policy using s_test = [–1; 1; 0; 0; 0; 0], and also call 4 batch_nse.m to compute r_nse, f_nse, and V_nse on the most-recent minibatch, and print those three variables alongside the G from test_policy. If your f and r functions and your mu network are correct, then you should get G = –92.8 (to one decimal place) the first time you call test_policy. If you program the learning correctly, you should have G > –65 and r_nse, f_nse, and V_nse < 0.01 by 1000 rollouts. Please name your variables as in these pages, the notes, and the sample code, e.g. q, psi, zeta, M, GAMMA, s, a, mu, f_est, f_tgt, r_est, V_est, V_tgt, eta_mu, eta_V, tau, etc. Submit an m-file called YOURINITIAL_YOURSURNAME_A3.
In what follows, INIT and SURNAME stand for your given-name initial and your surname in upper-case letters, e.g. for me, D and TWEED. And init and surname stand for your initial and surname in lower-case letters, e.g. d and tweed.Your job is to use backpropagation with adam to train a network to recognize the handwritten digits in the MNIST data set. The network should be a 4-layer mapnet where the two middle layers are 28-by-28 maps with field-fractions of 0.25. Please construct your network so that each layer projects to the next one (as in create_net.m), but layer 2 also projects directly to layer 4 through weight matrix W{4} — so W{4} is a 10-by-1568 matrix with input [y{2}; y{3}]. And please make the connections from both middle layers to the output layer dense (that is, fully connected, not maplike). In the two middle layers, use the following activation function, which we’ll call relog: y{l}i = max( 0, sgn(v{l}i) log( 1 + |v{l}i| ) ). Make the final layer of your network affine, but then apply a softmax function to the network’s output signal: yi = exp(y{4}i) / Σj exp(y{4}j).From the resulting y vector and the one-hot label vector y*, compute the error and loss as e = y – y* and L = e T e/2. To make your mapnet, write an m-file called init_surname_create_ mapnet, analogous to the posted m-file create_net. All the vectors and matrices associated with your mapnet should be initialized inside init_ surname_create_mapnet. In particular, that m-file should set all biases b to 0, and should initialize each neuron’s weight by drawing it from a normal distribution with a mean of 0 and standard deviation of sqrt(2/N), where N is the number of the neuron’s inputs, i.e. the number of upstream neurons projecting to it directly. For learning, write m-files init_surname_forward_relog and init_surname_backprop_relog, analogous to the posted m-files forward_relu and backprop_relu but modified to use the relog function instead of relu and to take into account the direct projection from layer 2 to layer 4. Please compute the softmax outside your forward_relog function, so your code calls your forward_relog and then applies softmax to its output. And compute the derivative ∂L/∂v{4} outside your backprop_ relog function and use that derivative as the input to backprop_relog. Implement adam using the posted m-file adam.m, and set the adam η = 0.001 and the other adam hyperparameters to their usual values. You’ll find the MNIST data set in the file MNIST.mat, along with MNIST_key.m, which explains the format of those data. Load the data set as in MNIST_key.m. Feed the images to the network in minibatches of 100 each. If you can avoid it, don’t write any code, anywhere in your files, that uses a loop or list to go through the minibatch example by example; instead, for concision and speed, write your code so that all operations are done on the whole minibatch array at once, as in the posted sample code. Run 10 epochs — that is, 10 passes through all 60,000 training examples — with 600 minibatches in each epoch. At the start of each epoch, shuffle the data, as in MNIST_key.m (this shuffling can aid learning, as it supplies a different series of 600 minibatches to the network in different epochs).After each epoch, run your network on the entire test set (in one big “minibatch” of all 10,000 test examples), count the number of incorrect guesses, and display that number in the Command Window. If all goes well, then on most runs, the number of incorrect guesses should be 400 or better after one epoch, and should fall below 160 at least once in the first 10 epochs. Submit an m-file called INIT_SURNAME_A2.m, along with init_ surname_create_mapnet.m, init_surname_forward_relog.m, and init_ surname_backprop_relog.m.
Your job is to simulate a simplified, two-joint arm and design a policy with Hurwitz desired dynamics and internal feedback that makes the arm reach to and track a moving target. In continuous time, the arm’s dynamics (apart from joint-bounding, described below) are described by the differential equation, 1 2 2 3 2 2 2 1 2 2 2 3 2 2 3 1 ( ) ( , ) 2 cos( ) cos( ) ( ) sin( ) , cos( ) 0 M q q q q q ψ ψ q ψ ψ q q q q q ψ q q ψ ψ q ψ q where q is the configuration vector containing the shoulder and elbow angles in that order, q and q are velocity and acceleration, τ is the vector of torques at the two joints, and ψ1, ψ2, and ψ3 are constants: ψ1 = 1.88, ψ2 = 0.6, and ψ3 = 0.48. The torque vector τ itself depends on the neural command or action according to the equation, a , where ζ = 0.75. 2 In your code, use the name torque for the τ vector, and zeta for ζ, and integrate the dynamics using Euler’s method, with ∆t = 0.01 s. You should also bound the joint angles to keep them inside their motion ranges. That is, define q_min = [-2; 0], q_max = [1.5; 2.5], and after Euler integration write: q = max(q_min, min(q_max, q)); q_vel = q_vel – q_vel.*( (q == q_max).*(q_vel > 0) + … (q == q_min).*(q_vel < 0) ); Program the arm’s reaching target (the desired joint-angle vector q*) so that it jumps, every 1 s, to a new, random location inside the arm’s joint ranges and then glides at a random, constant velocity between jumps, with speeds ranging between 0 and about 2.5 to 3.5 rad/s, always staying inside the joint ranges, as shown by the dotted lines in the plot on the next page. Choose Hurwitz desired dynamics that get both joints to their target values within about 0.75 s or quicker and then keep them on the gliding target until the next jump. Design a policy that implements those desired dynamics and keeps both elements of a smaller than about 25,000 in absolute value (you needn’t prove mathematically that a never crosses those bounds — just look at your simulations to check that it usually stays in about that range). Your agent must rely on internal feedback. The only things it can know directly are its own action a, the position and velocity of the target, and the forms of the dynamics equations including the joint bounds and where the various constants and variables (ψ1, ψ2, ψ3, q, q (1) , a and so on) plug into those equations. Give your agent exactly correct estimates of ψ1, ψ2, ψ3, and ζ, and have it use internal simulations to estimate q and q (1) and any other necessary variables. Give the agent correct initial estimates of those variables, and reset those estimates to their correct values every time the target jumps. You can use the sample code Saccadic_internal.m as a partial guide, but your agent should not assume that actual velocity equals desired velocity. 3 The agent doesn’t know anything else about the target except its current position and velocity. For instance, it doesn’t know that the target will jump at 1-s intervals or maintain a constant velocity between jumps. In the handout notes, we derived policies using scalar operations, but here you will need vectors and matrices, because q has more than one element. For vector and matrix operations anywhere in your code, use Matlab’s built-in functions, e.g. if you need to multiply A and B or invert or transpose A, then write A*B or inv(A) or A’ — don’t write out the complicated, element-wise formulae for these operations. Display your results in a figure with 2 panels. In the upper panel, plot shoulder and elbow angles (as red and blue solid lines) and their targets (as red and blue dotted lines) versus time, with y-axis limits set using ylim(1.05*[min(q_min), max(q_max)]). In the lower panel, plot both elements of a versus time. Simulate 10 s in all, or in other words 10 reaches to 10 targets. If all goes well, your upper-panel plot should look something like this: Please name your variables as in these pages, the notes, and the sample code: q, q_star, q_vel, q_acc, psi, zeta, M, GAMMA, a, and so on. Submit an m-file called Yourgivennameinitial_Yoursurname_A1, as for example D_Tweed_A1.m.
The project can be done in groups of 2. If you decide to do the project in a group, make sure that you request us to create a single group repository on GitLab. This is the final course project. For the project you will need to: • Augment your code from Assignment 4 in the way that is described below. • Quantitatively analyze your software for various kinds of inputs. • Write a brief report (≈ 5 pages, 11 pt font, reasonable margins) with your analysis. Your report must be typeset in LATEX, and must be in PDF. You should augment your code from Assignment 4 in the following ways. • Make it multithreaded. You should have at least 4 threads: one for I/O, and one each for the different approaches to solve the minimum vertex cover problem. • Implement the following two additional ways to solve MIN-VERTEX-COVER, in addition to the REDUCTION-TO-CNF-SAT approach you had in Assignment 4. (We will call your approach from Assignment 4, CNF-SAT-VC.) 1. Pick a vertex of highest degree (most incident edges). Add it to your vertex cover and throw away all edges incident on that vertex. Repeat till no edges remain. We will call this algorithm APPROX-VC-1. 2. Pick an edge ⟨u, v⟩, and add both u and v to your vertex cover. Throw away all edges attached to u and v. Repeat till no edges remain. We will call this algorithm APPROXVC-2. Inputs As input, use the output of /home/agurfink/ece650/graphGen/graphGen on eceubuntu. That program generates graphs with the same number of edges for a particular number of vertices, but not necessarily the same edges. Note that you can store its output in a file and use the file on other machines. Output Given a graph as input, your program should output the vertex cover computed by each approach in sorted order. That is, give the following input: V 5 E {,,,,} 1 The output from your program should be: CNF-SAT-VC: 3,5 APPROX-VC-1: 3,5 APPROX-VC-2: 1,3,4,5 That is, the name of the algorithm, followed by a colon ’:’, a single space, and then the computed result as a sorted sequence of vertices, separated by commas. Analysis You should analyze how efficient each approach is, for various inputs. An input is characterized by the number of vertices. “Efficient” is characterized in one of two ways: (1) running time, and (2) approximation ratio. We characterize the approximation ratio as the ratio of the size of the computed vertex cover to the size of an optimal (minimum-sized) vertex cover. For measuring the running time, use pthread getcpuclockid(). For an example of how it is used, see http://www.kernel.org/doc/man-pages/online/pages/man3/pthread getcpuclockid.3.html. For measuring the approximation ratio, compare it to the output of CNF-SAT-VC, which is guaranteed to be optimal. Your objective is to measure, for various values of |V | (number of vertices), for the graphs generated by graphGen, the running time and approximation ratio. You should do this by generating graphs for |V | ∈ [5, 50] using that program, in increments of 5. That is, graphs with 5, 10, 15, . . . , 50 vertices. You should generate at least 10 graphs for each value for |V |, compute the time and approximation ratio for each such graph. You should measure the running time for at least 10 runs of each such graph. Then, you should compute the mean (average) and standard deviation across those 100 runs for each value of |V |. For the approximation ratio, if there is any random component (e.g., which edges you choose, for APPROX-VC-2), then you should measure that multiple times as well for each graph. You might find the optimal approach (CNF-SAT-VC) difficult to scale to large number of vertices. For these instances, you can use a timeout to avoid waiting for its result and produce CNF-SAT-VC: timeout in the output. CNF-SAT-VC can be scaled significantly by improving the encoding. Should you decide to improve the encoding, bonus points will be given for scaling to larger instances. Make sure to describe the imporvements you made to the encoding in the report. Points will be deducted for encodings that are scalable but are either incorrect or unexplained. Report The main part of your report are graphs (plots) corresponding to the data you generate as described in the “Analysis” section above. One way to show the output is to have two plots: one for running times and the other for approximation ratio. The horizontal axis is the number of vertices. You should plot the mean for each value of |V | for which you made measurements, and the standard deviation as an errorbar1 . An example of a possible plot is shown in Figure 1. The remainder of your report should be reasoning about your plots. That is, you should explain why your plots look the way they do. For example, if there is a “spike” in the approximation ratio for some value of |V | for one of the approaches, you should explain why there is such a spike. You should also explain apparent trends. For example, if, for one of the approaches, the running time seems to increase linearly with |V |, you should reason about why that is happening. 1You can use yerrorbar in gnuplot to draw an error bar: http://gnuplot.sourceforge.net/docs 4.2/node262.html 2 Figure 1: Example plot, generated using gnuplot. The error bars for Approx-2 are not visible because the standard deviation is small. Marking We will mark by: (1) Trying some inputs and checking your output, (2) inspecting your code to make sure that you are using pthreads correctly, and, (3) reading your report. • Marking script for compile/make etc. fails: automatic 0 • Your program runs, awaits input and does not crash on input: + 20 • Correctly implemented 2 new algorithms: + 20 each, total + 40 • Generated plots: + 20 • Report: + 20 • Bonus: + 10 CMake As discussed below under “Submission Instructions”, you should use a CMakeLists.txt file to build your project. We will build your project using the following sequence: cd project && mkdir build && cd build && cmake ../ You can assume that MiniSat will be placed in directory project/minisat. Your submission should only include your own code. If your code is not compiled from scratch (i.e., from the C++ sources), you get an automatic 0. Unlike for the assignments, you must create the CMakeLists.txt file on your own. You can use a CMakeLists.txt file from previous projects or from the course examples provided on GitLab. 3 Submission Instructions You should place all your files in the project directory in your GitLab repository. The directory should contain: • All your C++ source-code files. • A CMakeLists.txt, that builds your C++ executable ece650-prj. • A file user.yml that includes your name, WatIAM, and student number of all the team members. Note that WatIAM is the user name for your Quest account, e.g. agurfink, and a student number is an 8-digit number, e.g. 20397238. If you have done the assignment as a group, the information for both members of the group should be included in the user.yml. • A file named “report.pdf” with your report. See README.md for any additional information. The submitted files should go to project directory in your repository.
For this assignment, you are to augment your code from Assignment 2 to solve the minimal Vertex Cover problem for the input graph. Your approach is based on a polynomial time reduction to CNFSAT, and use of a SAT solver. The following are the steps you should take for this assignment. SAT Solver We will be using MiniSat SAT solver available at https://git.uwaterloo.ca/ece650-common/ minisat.git MiniSat provides a CMake build system. You can compile it using the usual sequence: cd PROJECT && mkdir build && cd build && cmake ../ && make The build process creates an executable minisat and a library libminisat.a. You will need to link against the library in your assignment. Play around with it. Sample files are available in the https://git.uwaterloo.ca/ece650-common/ minisat-example repository on GitLab. Incorporate SAT Create a polynomial reduction of the decision version of VERTEX COVER to CNF-SAT. We have discussed the reduction in class. It is also available under the name a4 encoding.pdf on LEARN. You are allowed to use your own reduction provided it is sound and polynomial-time. Implement the reduction and use minisat as a library to solve the minimum VERTEX COVER problem for the graphs that are input to your program (as in Assignment 2). As soon as you get an input graph via the ’V’ and ’E’ specification you should compute a minimum-sized Vertex Cover, and immediately output it. The output should just be a sequence of vertices in increasing order separated by one space each. You can use qsort(3) or std::sort for sorting. Assuming that your executable is called ece650-a4, the following is a sample run of your program: $ ./ece650-a4 V 5 E {,,,,,} 4 5 1 The lines starting with V and E are the inputs to your program, and the last line is the output. Note that the minimum-sized vertex cover is not necessarily unique. You need to output just one of them. Marking We will try different graph inputs and check what vertex cover you output. We will only test your program with syntactically and semantically correct inputs. • Marking script for compile/make etc. fails: automatic 0 • Your program runs, awaits input and does not crash on input: + 20 • Passes Test Case 1: + 25 • Passes Test Case 2: + 25 • Passes Test Case 3: + 25 • Programming style: + 5 CMake As discussed below under “Submission Instructions”, you should use a CMakeLists.txt file to build your project. We will build your project using the following sequence: cd a4 && mkdir build && cd build && cmake ../ If your code is not compiled from scratch (i.e., from the C++ sources), you get an automatic 0. Submission Instructions You should place all your files in your GitLab repository in directory a4. The directory should contain: • All your C++ source-code files. • A CMakeLists.txt, that builds your C++ executable ece650-a4. • A file user.yml that includes your name, WatIAM, and student number. Note that WatIAM is the user name for your Quest account, e.g. awasef, and a student number is an 8-digit number, e.g. 20397238. See README.md for any additional information. You should assume that MiniSat will be placed in directory a4/minisat. Your submission should include only your own code (including code provided by us in the skeleton). If your code is not compiled from scratch (i.e., from the C++ sources), you get an automatic 0. 2 A Polynomial-Time Reduction from VERTEX-COVER to CNF-SAT A vertex cover of a graph G = (V, E) is a subset of vertices C ⊆ V such that each edge in E is incident to at least one vertex in C. VERTEX-COVER is the following problem: • Input: An undirected graph G = (V, E), and an integer k ∈ [0, |V |]. • Output: True, if G has a vertex cover of size k, false otherwise. CNF-SAT is the following problem: • Input: a propositional logic formula, F, in Conjunctive Normal Form (CNF). That is, F = c1 ∧ c2 ∧ . . . ∧ cm, for some positive integer m. Each such ci is called a “clause”. A clause ci = li,1 ∨ . . . ∨ li,p, for some positive integer p. Each such li,j is called a “literal.” A literal li,j is either an atom, or the negation of an atom. • Output: True, if F is satisfiable, false otherwise. We present a polynomial-time reduction from VERTEX-COVER to CNF-SAT. A polynomial-time reduction is an algorithm that runs in time polynomial in its input. In our case, it takes as input G, k and produces a formula F with the property that G has a vertex cover of size k if and only if F is satisfiable. The use of such a reduction is that given an instance of VERTEX-COVER that we want to solve, (G, k), we use the reduction to transform it to F, and provide F as input to a SAT solver. The true/false answer from the SAT solver is the answer to the instance of VERTEX-COVER. Assuming the SAT solver works efficiently (for some characterization of “efficient”), we now have an efficient way of solving VERTEX-COVER. Furthermore, the satisfying assignment from the SAT solver can be used to re-construct the solution to VERTEX-COVER. The reduction Given a pair (G, k), where G = (V, E), denote |V | = n. Assume that the vertices are named 1, . . . , n. Construct F as follows. • The reduction uses n × k atomic propositions, denoted xi,j , where i ∈ [1, n] and j ∈ [1, k]. A vertex cover of size k is a list of k vertices. An atomic proposition xi,j is true if and only if the vertex i of V is the jth vertex in that list. • The reduction consists of the following clauses – At least one vertex is the ith vertex in the vertex cover: ∀i ∈ [1, k], aclause(x1,i ∨ x2,i ∨ · · · ∨ xn,i) – No one vertex can appear twice in a vertex cover. ∀m ∈ [1, n], ∀p, q ∈ [1, k]withp < q, aclause(¬xm,p ∨ ¬xm,q) In other words, it is not the case that vertex m appears both in positions p and q of the vertex cover. – No more than one vertex appears in the mth position of the vertex cover. ∀m ∈ [1, k], ∀p, q ∈ [1, n]withp < q, aclause(¬xp,m ∨ ¬xq,m) – Every edge is incident to at least one vertex in the vertex cover. ∀⟨i, j⟩ ∈ E, aclause(xi,1 ∨ xi,2 ∨ · · · ∨ xi,k ∨ xj,1 ∨ xj,2 ∨ · · · ∨ xj,k) The number of clauses in the reduction is k + nk2 + kn2 + |E|. 3
For this assignment, you are to: • Modify the output format of your Python script from Assignment 1 to match the input format of your C++ program from Assignment 2. • Modify your C++ program from Assignment 2 to output the input graph on standard output. • Write a program in C++ to generate random input for your Python script. • Write a driver program in C++ that uses Inter-Process Communication (IPC) to link the output of the random input generator to the input of the Python script, and the output of the Python script to the input of the C++ program from Assignment 2. Resources The online book, “Operating Systems: Three Easy Pieces”, will be useful for this assignment and beyond. For this assignment, you might find Chapter 4 on Processes and Chapter 5 on Process API the most useful. The book is available online at http://pages.cs.wisc.edu/∼remzi/OSTEP/. You might also want to consult “Advanced Linux Programming”, especially Chapter 3 on Processes and Chapter 5.4 on Pipes. The book is available from https://github.com/MentorEmbedded/ advancedlinuxprogramming/blob/gh-pages/alp-folder/advanced-linux-programming.pdf Sample Run You should name the driver’s executable ece650-a3. In the following, we assume that “$” is the shell command-prompt. $ cd build $ cmake ../ $ make install $ cd ./run/bin $ ./ece650-a3 -s 5 -n 4 -l 5 V 8 E {,,,,,,} s 3 5 3-1-5 In the above, the first three lines make your executable, and run the driver program with some command-line arguments. Then, the lines “V = …”, “E = …”, and “3-1-5” are output. The input the user provided to your program via stdin is “s 3 5”. 1 Input, Output, and Error Your program should take input from stdin, and output to stdout. Errors should be output to stderr. Errors should always start with “Error:” followed by a brief description. All your processes should terminate gracefully (and quietly) once you see EOF at stdin. Your program should not generate any extraneous output; for example, do not print out prompt strings such as “please enter input” and things like that. As the example above indicates, there are two kinds of input the user provides. One is via the command-line arguments, with switches such as -s and -n . This is done only once, when your program is first started. The other type of input is the ’s’ command on stdin, which may be issued repeatedly, just as in Assignment 2. For the ’s’ command, your program should output a shortest path. We will not test your program for format errors in the input. That is, the command-line arguments, whenever specified, will be formatted correctly, and the s input will also be formatted correctly. Of course, we may omit command-line arguments (see below for what to do in such cases), and specify vertex IDs with s that do not exist, or between whom a path does not exist. The latter two cases should cause your program to report an error. Marking • Does not compile/make/crashes: automatic 0 • Your program runs, awaits input and does not crash on input: + 20 • Run in default mode: + 15 • Error check arguments: + 5 • Test functionality: + 20 • Test rgen: + 20 • Replace rgen: + 20 CMake As discussed below under “Submission Instructions”, you should use a CMakeLists.txt file to build your project. We will build your project using the following sequence: cd a3 && mkdir build && cd build && cmake ../ && make install If your code is not compiled from scratch (i.e., from the C++ sources), you get an automatic 0. Note that we are using make install instead of make. The install target instructs make to copy all of the binaries (both Python and C++) into directory ./run/bin. This ensures that all the executable files (including the Python program) are in the same directory. Submission Instructions You should place all your files in the a3 directory in your GitLab repository. The directory should contain: • All your C++ and Python source-code files. IMPORTANT NOTE: the executable for your random generator must be named rgen. The reason is that part of our testing will replace your generator with ours. • A CMakeLists.txt, that builds all of the C++ executables: rgen, ece650-a2, and ece650-a3. • A file user.yml that includes your name, WatIAM, and student number. See README.md for any additional information. The submitted files should be in the a3 directory of the master branch of your repository. 2 File names There are two file names that are important. One is your main executable. This must be named ece650-a3. Our marking script will simply try and run this after building your project. The other executable file whose name is important is your random input generator. The executable for this must be named rgen. The reason is that for some of our tests, we will replace your rgen with ours. Things to be done Python script You should edit your Python script from Assignment 1 so that its output which is the specification of the graph is formatted to match the input format of your C++ program from Assignment 2. Think of it as though your Python script provides command-line input to the C++ program. The way to do this is to simply map each vertex to an index in [1, n] (where n is the number of vertices), and rename your edges accordingly. Also, the only output that your Python script should produce to stdout is in response to “gg”, for which it outputs the specification of the graph (i.e., V and E). Error output should go to stderr. Cpp program from Assignment 2 You should edit it to print the graph that it has read before accepting an s command. This is necessary so that you know what graph has been produced and which vertices are available. Random input generator Your random input generator rgen should generate random street specifications as input for your Python script. It takes four command-line arguments. All are optional. • -s k — where k is an integer ≥ 2. The number of streets should be a random integer in [2, k]. If this option is not specified, you should use a default of k = 10; that is, the number of streets should be a random integer in [2, 10]. • -n k — where k is an integer ≥ 1. The number of line-segments in each street should be a random integer in [1, k]. Default: k = 5. • -l k — where k is an integer ≥ 5. Your process should wait a random number w seconds, where w is in [5, k] before generating the next (random) input. Default: k = 5. • -c k — where k is an integer ≥ 1. Your process should generate (x, y) coordinates such that every x and y value is in the range [−k, k]. For example, if k = 15, all of your coordinate values should be integers between −15 and 15. Default: k = 20. Your program should generate a specification of streets in the format that your Python script expects (see Assignment 1). You can name the streets whatever you want. You should ensure that your input does not have errors. For example, if you generate a line-segment that overlaps with a line-segment (across all streets) generated earlier, you should regenerate that line-segment. Similarly, you should not have any zero-length line segments. Also, note that your random generator could go into an infinite loop looking for a valid specification. You should disallow this from happening by limiting the number of tries. That is, if your random generator fails to generate a valid specification for a continuous A number of attempts, it should exit() with an error message reported on stderr. A reasonable A to adopt may be 25. Whatever A you adopt, your error message should mention it. That is, your error message should be something like, “Error: failed to generate valid input for 25 simultaneous attempts”. Before adding a new street graph specification to your Python script, your generator should issue “rm” commands to your Python script to remove all previous streets and replace them with the new street specification. 3 After generating the input, the generator must issue the “gg” command. Thus, a typical interaction of the random generator is as follows: 1. issue enough rm commands to clear any existing street database; 2. issue add commands to add new streets satisfying the specification; 3. issue a gg command; 4. wait for specified number of seconds and repeat. You must use /dev/urandom as the source of your random data. Also, see under “Submission Instructions” how your executable for the random generator must be named. Driver Your driver program has the overall control. You have at least three programs that run concurrently: (1) the random generator, (2) your Python script from Assignment 1, and, (3) your program from Assignment 2 that takes a graph-specification as input and computes shortest paths. Your driver program should fork() two processes and exec() two of those programs, and then exec() the third (so it turns into a process that corresponds to the third program). It should set up all IPC appropriately beforehand. It should send normal output to stdout, error output to stderr, and take input from stdin. As we mention above, the only input it takes are “s” commands, that ask for a shortest path between vertices. Gotcha warning: note that I say “you have at least three programs that run concurrently.” You may need more than three that run concurrently. (Why?) Sample code The skeleton repository for the assignment contains some sample code. You should replace all sample code with your own code. Including replacing the Python code with your solution from Assignment 1, and C++ code from your solution to Assignment 2. 4
• For this assignment, you need to write a program that takes input till it sees an EOF. • One of the kinds of input contributes towards the specification of an undirected graph. • Another kind of input asks you to print out a shortest-path from one vertex to another in the current graph. • Write your code in C++. Ensure that it compiles with the C++compiler on eceubuntu, and runs on eceubuntu. Resources The book, “Introduction to Algorithms”, by Cormen, Leiserson, Rivest & Stein, may be useful to you. The 2nd ed. is available electronically at the library via lib.uwaterloo.ca. Section 22.1 discusses how a graph may be represented. For determining a shortest-path, you can use a simple algorithm, such as Breadth-First Search (22.2), or Bellman-Ford (24.1). Sample Run Assume that your executable is called ece650-a2. In the following, “$” is the command-prompt. $ ./ece650-a2 V 15 E {,,,,,,} s 2 10 2-8-10 V 5 E {,,,,} s 5 1 5-2-3-1 Input, Output, and Error Your program should take input from standard input, and output to standard output. Errors can also be output to standard output, but should always start with “Error:” followed by a brief description. Your program should terminate gracefully (and quietly) once it sees EOF. Your program should not generate any extraneous output; for example, do not print out prompt strings such as “please enter input” and things like that. As the example above indicates, one kind of input is the specification of a set of vertices V, and set of edges E of the undirected graph. The specification of a set of vertices starts with ‘V’, followed 1 by a space, followed by an integer greater than one, all in one single line. If the integer that follows the V is i, then we assume that the vertices are identified by 1, . . . , i. The specification for a set of edges starts with ‘E’. It then has a space, followed by the set of edges in a single line delimited by ‘{’ and ‘}’. The two vertices of an edge are delimited by ‘’ and separated by a comma. The edges in the set are also separated by a comma. There are no whitespace characters within the { }. The only other kind of input starts with an ‘s’. It asks for a shortest path from the first vertex to the second that is specified after the s. The s is followed by a space, a vertex ID, another space, and a second vertex ID. The lines 2-8-10 and 5-2-3-1 above are outputs of the s commands that immediately precede them. The output comprises vertex IDs separated by ‘-’, with no whitespace within. If a path does not exist between the vertices, you should output an error. The graph is specified by the specification of the set of vertices V followed by the set of edges E, in that order. V and E always occur together. There is no relationship between subsequent graph specifications; when a new V specification starts, all previous information can be forgotten. After the specification of the graph there can be zero or more shortest-path queries s. For each s query, only one shortest path should be output; multiple shortest paths might exist and an arbitrary choice can be made. We will not test your program for format errors in the input. That is, our input will be perfectly formed. However, we will test your program for other kinds of errors. For example, if we have ‘V 5’, we may try to specify an edge , for a vertex 10 that does not exist. Similarly, we may ask for a shortest path to a vertex that does not exist. We may also ask for a shortest path when the vertices exist, but a path does not exist between them. Errors should be output in such cases. Marking Your output has to perfectly match what is expected. You should also follow the submission instructions carefully. It discusses how to name your files, how to submit, etc. The reason is that our marking is automated. • Does not compile/make/crashes: automatic 0 • Your program runs, awaits input and does not crash on input: + 20 • Passes Test Case 1: + 20 • Passes Test Case 2: + 20 • Passes Test Case 3: + 15 • Correctly detects errors: + 20 • Programming style: + 5 CMake As discussed below under “Submission Instructions”, you should create a CMakeLists.txt file to build your project. We will build your project using the following sequence: cd a2 && mkdir build && cd build && cmake ../ && make at the top level of your repository. If your code is not compiled from scratch (i.e., from the C++ sources), you get an automatic 0. Submission Instructions You should place all your files at the a2 directory in your GitLab repository. The directory should contain: • All your C++ source-code files; you may use further sub-directories if you wish. • A CMakeLists.txt, that builds a final executable named ece650-a2. 2 • A file user.yml that includes your name, WatIAM, and student number. See README.md for any additional information. The submitted files should be in a2 directory in the master branch of your repository. 3
This is the first in a series of assignments that is part of a single large project. The project is to help the local police department with their installation of security cameras at traffic intersections. You will solve a particular kind of optimization problem, called the Vertex Cover problem, in this context. The idea is for the police to be able to minimize the number of cameras they need to install, and still be as effective as possible with their monitoring. For this assignment, you need to: • Take as input a series of commands that describe streets. • Use that input to construct a particular kind of undirected graph. • Write your code in Python (version 3). • Ensure that it works on eceubuntu.uwaterloo.ca. (You are allowed to use only those Python libraries that are already installed on those machines. You are not allowed to install any new libraries.) (Use eceterm.uwaterloo.ca to log-in from off-campus; then follow the instructions to connect to eceubuntu.) Sample Input The input comprises lines each of which specifies a command. There are 4 kinds of commands. (1) add a street, (2) modify a street, (3) remove a street, and, (4) generate a graph. Here is an example of how your program should work. Visualizing this example using the Cartesian coordinate system may help you understand what’s going on. 1 add “Weber Street” (2,-1) (2,2) (5,5) (5,6) (3,8) add “King Street S” (4,2) (4,8) add “Davenport Road” (1,4) (5,8) gg V = { 1: (2,2) 2: (4,2) 3: (4,4) 4: (5,5) 5: (1,4) 6: (4,7) 7: (5,6) 8: (5,8) 9: (3,8) 10: (4,8) } E = { , , , , , , , , } mod “Weber Street” (2,1) (2,2) gg V = { 2: (4,2) 5: (1,4) 6: (4,7) 8: (5,8) 10: (4,8) } E = { , , , } rm “King Street S” gg V = { } E = { } 2 Commands • add is used to add a street. It is specified as: “add “Street Name” (x1, y1) (x2, y2) . . . (xn, yn)”. Each (xi , yi) is a GPS coordinate. We interpret the coordinates as a poly-line segment. That is, we draw a line segment from (xi , yi) to (xi+1, yi+1). You are allowed to assume that each xi and yi is an integer. (Note, however, that the coordinates of an intersection may not be integers.) • mod is used to modify the specification of a street. Its format is the same as for add. It is a new specification for a street you’ve specified before. • rm is used to remove a street. It is specified as “rm “Street Name””. • gg causes the program to output the corresponding graph. Input and Output Your program should take input from standard input. Your program should output to the standard output. Error should be output to standard error. You can use exceptions in your code to catch errors. Errors The above example is that of a “perfect” user — someone that did not make any mistakes with specifying the input. You should account for errors in the input. If a line in the input is erroneous, you should immediately output an error message. The format of the message is to be the string “Error:” followed by a brief descriptive message about the error. For example: Error: `mod’ or `rm’ specified for a street that does not exist. Your program should recover from the error as well. That is, your program should reject the errorneous line, but continue to accept input. Your program should not crash because of an error. Any erroneous input we try will be of a relatively benign nature that mimics honest mistakes a user makes. We will not try malicious input, such as unduly long lines or weird control characters. The Output Graph There is a vertex corresponding to: (a) each intersection, and, (b) the end-point of a line segment of a street that intersects with another street. An example of (a) from above is Vertex 3. An example of (b) is Vertex 1. The identity of a vertex can be any string of letters or integers (but no special characters). For example, v1xyz is acceptable as the identity of a vertex, but not v1 !!#xyz. (The space is unacceptable, as are ’!’ and ’#’. There is an edge between two vertices if: (a) at least one of them is an intersection, (b) both lie on the same street, and, (c) one is reachable from the other without traversing another vertex. An example from above is the edge ⟨1, 3⟩, which connects the end-point of a line segment to an intersection. Another example is the edge ⟨3, 6⟩, which connects two intersections. Marking Your output has to perfectly match what is expected. You should also follow the submissions instructions carefully. The reason is that our marking is automated. • Does not compile/make/crashes: automatic 0 • Your program runs, awaits input and does not crash on input: + 20 • Passes Test Case 1: + 20 • Passes Test Case 2: + 20 • Passes Test Case 3: + 15 • Correctly detects errors: + 25 3 • Programming style: + 0 for this assignment, but will be > 0 for future assignments. Get Skeleton Code from Upstream To get the code, remember to pull from upstream/master. Python Resources If you’re looking for some practice using and understanding Python3 you may wish to use http: //pythontutor.com/. There is one example one the main page that you can step through lineby-line. More examples can be found on the live online editor page (http://pythontutor.com/ visualize.html). The generated visualization of the code may be useful to you. The python tutor webpage also has code visualization for C++which you may find useful for the next assignment. If you have Python code on your computer that you wish to step through in a similar way, either line-by-line or some other way, you can run the code using the Python debugger. The Python debugger (pdb) can be invoked on the command line by providing the flag and argument as follows: python3 -m pdb Documentation on how to use the debugger can be found on https://docs.python.org/3/ library/pdb.html. Submission Instructions Your code can be in multiple Python source files. The main python file should be a1/a1ece650.py. The assignment should be submitted as a master branch on YOUR GitLab in directory a1. Don’t forget to enter your student identification in a1/user.yml and follow any additional instructions in a1/README.md.
SOCS0031: Introduc1on to Economics II Problem Set 2023-24 Instruc1ons: • This problem set has 4 pages with 3 ques5ons and an appendix containing the ar5cles for the two op5ons in ques5on 3. • Please answer the 2 core ques1ons AND choose 1 of the two op1ons presented in ques1on 3. You are required to submit answers to 3 ques1ons in total. • Please note that answering both op5onal ques5ons will not confer any addi5onal advantage, as only the first answer provided will be considered for your final grade. Ensure that you address every part of the ques5ons you choose to answer. Incorrect answers will not result in point deduc5ons; however, you may receive par5al credit for incomplete but par5ally correct responses. • This problem set is scored out of 100 points and counts as 30% of your final mark (the other 70% of your final mark will come from a final exam taking place in Term 3.) • Carefully read each ques5on, paying close aLen5on to any assump5ons or hints provided, as they are intended to assist you. For addi5onal informa5on regarding the assessment or submission procedures, refer to the module handbook and the Moodle plaOor m. Please refer to the Guidance on the use of genera5ve AI tools in assessment in the Academic integrity content bar on Moodle if you are using AI. • You are reminded that this is intended to be individual work; do not work with others in comple5ng your work. More informa5on on UCL’s academic integrity policy can be found in Sec5on 4.2 of Module handbook and at hLps://www.ucl.ac.uk/students/exams-and-assessments/academic- integrity. • Submit your answers in one, single file either .doc/.docx or .pdf format via Moodle by 13:00 on 20 February, 2024. Core ques1on 1 Distor1onary Policies in the Labour Market Consider a labour market where firms demand labour and workers supply labour: a) (5 marks) First, let’s analyse the case where the labour market is perfectly compe55ve where no agent, workers or employers, has market power. Illustrate the compe55ve equilibrium and iden5fy the total welfare in a graph. Label the compe55ve market wage as w* and the compe55ve quan5ty of labour as l*. b) (5 marks) The government is concerned that w* is not a “living wage” and would like to regulate the labour markets to correct this. Explain briefly at least two policy tools available to the government that could result in a “living wage”. Pick one of your chosen policies and illustrate graphically, using the graph you have drawn in part a), what the impact of this policy is on wage, employment, producer, and consumer surplus. c) (7 marks) Now, consider the case of a monopsonist employer. Explain in what circumstances such a market failure can arise. Show in a graph the market equilibrium in a monopsonist labour market. How does that compare to a compe55ve equilibrium like the one you iden5fied in part a) in rela5on to w*, l* and welfare? d) (8 marks) The government considers the introduc5on of a minimum wage policy to address the market inefficiency of monopsonist employers. Illustrate graphically the impact of this policy on the market equilibrium (wage, level of employment) and welfare. e) (5 marks) What informa5on is necessary for the government to set a minimum wage policy? Discuss briefly. Core ques1on 2 Compe11ve advantage and tariffs In this exercise we look at two countries, UK and Portugal, that produce and consume two goods, cheese and wine. England has 100,000 units of labour. Portugal has 50,000 units of labour, that are able to produce both goods according to this table: a) (2 marks) Does any country have absolute advantage in producing both goods? Which one? b) (4 marks) Draw the produc5on possibility fron5ers of both countries under autarky. What is the expected rela5ve price of wine in terms of cheese in the two countries in isola5on, without interna5onal trade? c) (4 marks) If the two countries are allowed to trade, what specialisa5on would each country adopt? d) (5 marks) Define the terms of trade (TOT) for the UK, and choose a single possible price that makes the UK strictly beLer off ajer trade rela5ve to autarky. e) (4 marks) Demonstrate the gains from trade ajer free trade is implemented. Illustrate this with an appropriate graph of your choice using the price you selected in d). f) (6 marks) The UK government decides to impose a tariff on its imported good. In a new graph, illustrate the impact of the tariff on the price of wine, the quan5ty consumed and welfare in the UK. g) (5 marks) And finally, in a different graph, illustrate the impact of the tariff on welfare in Portugal. 3. (40 points) OPTIONAL QUESTION – ONLY PICK AND COMPLETE ONE OF THE TWO Either of these op5ons will ask you to write a short essay of 600 words. The essay must be wriLen in full sentences and must be well-structured: each paragraph should deal with one major issue or idea and that these ideas should be developed in a logical sequence. Please include a short introduc5on, which lays out your main message (i.e. the ques5on you will answer or the claim you will argue for throughout the essay). Wrap up your work with a short conclusion. Note: This is an essay. There is not a single good answer. Make sure you state any assump5ons you need for your analysis and that your arguments are consistent with those assump5ons. Usual rules for academic referencing apply as described at hLps://www.ucl.ac.uk/library/libraryskills-ucl/guides-and-elearning/references-cita5ons-and-avoiding-plagiarism. OPTION 1: The monopoly power of the App Store Read the short ar5cle by Adam Satariano and Tripp Mickle, “Apple Overhauls App Store in Europe, in Response to New Digital Law” at: hLps://www.ny5mes.com/2020/10/08/business/colleges-pandemic-market- failure.html Using a (graphical) model we learnt in class, capture the effects of market power of Apple in the market for apps. Show and discuss how the ruling of the European Union would impact consumer welfare. OPTION 2: Nega1ve externali1es in educa1on due to the pandemic. Read the short ar5cle by Sarah Cohodes and Susan Dynarski, “Colleges Are Fuelling the Pandemic in a Classic Market Failure” at hLps://www.ny5mes.com/2020/10/08/business/colleges-pandemic-market- failure.html Using a (graphical) model we learnt in class, capture the effects of the nega5ve externality alluded to in the ar5cle. What the impacts in terms of welfare? And graphically represent how one of the solu5ons suggested would impact the market.
A2: Critical Analysis Task Part 1: Write a 900 word reflection The development of AI has provided a powerful impetus for academic research, especially in summarizing complex academic articles. As an example, this article (Murali et al. 2021), which centers around the responsibility that different national media should have for climate change, conducts an AI simulation of a summary that critically explores the use of AI in generating scholarly summaries, and Kate Crawford's Maps of Artificial Intelligence and other scholarly writings further critically reflect on the use of AI for scholarly research potential advantages and problems (Crawford 2021, 1-21). AI-generated summaries can significantly reduce the time it takes for readers to understand the central arguments and evidence of an article, enabling students and researchers to quickly grasp the main arguments and evidence of complex articles. For students and researchers, this technology can serve as an initial step in understanding the article and provide them with a broad understanding of the content before diving into the text.AI-generated summaries as a method not only increase efficiency, but also simplify complex terminology, making it easier for readers to access the core knowledge of the article, and making the knowledge accessible to the general public.The application of AI can reduce human errors, such as miscalculations and omissions of information, in addition to increasing the efficiency of acquiring knowledge. At the same time, AI-generated article summaries will not be easily influenced by emotions and surroundings, so they may be more objective overall than articles summarized by people themselves. However, this time-efficient convenience that AI brings comes with obvious drawbacks. The primary drawback is whether AI is accurate in the summaries it generates. AI's current capabilities do not always accurately explain and summarize subtle academic arguments and theoretical frameworks. Considering that AI writing relies on training data, if the data is biased or inaccurate, the generated text may also be problematic, which can lead to distorted and misleading information (Jones 2023,503). In addition, due to the convenience and intelligence of AI, many students will choose to turn to AI to solve the problems they should have thought and studied independently, and over time, students will be too lazy to think two not to learn, which will lead to academic wastage. At the same time, excessive use of AI will also involve academic integrity issues. Therefore, over-reliance on artificial intelligence will most likely weaken students' critical thinking and reading skills and seriously affect the quality of their degrees. According to Crawford's article, it is extremely clear that the application of AI in academia may not only be a technological issue, but also closely related to the ethical, political, and academic environment (Crawford 2021, 1-21). In layman’s terms, the development of AI may also bring ethical and moral issues. For example, AI may directly use other people's articles when generating academic summaries, which involves important issues related to intellectual property rights, ethics and fair access, and violates authors' intellectual property rights. As a result, the application of AI may lead to an increased likelihood of plagiarism in scholarship and devaluation of original scholarship (Ayodele et al. 2023, 1-21).The development and application of AI technology consumes a great deal of human, material, and financial resources and breaks down the traditional form. of academic research, which triggers the questioning of the continued development of AI in academia and the fairness of academic research. Therefore, one should view AI as an aid to academic research, a tool that needs to be utilized appropriately. While its efficiency and convenience are undeniable, vigilance must be maintained to prevent information bias and errors, and to avoid creating a dependence on AI that weakens students' critical thinking and reading skills. Interacting with AI to generate abstracts should be a starting point for academic research, rather than viewing AI-generated abstracts as a substitute for in-depth reading and critical analysis. In addition, the low level of interpretability, data bias, and ethical issues of AI technologies pose significant risks to users, developers, humans, and society. Crawford and others therefore emphasize the responsible integration of AI technologies at an ethical level and the careful consideration of their wider implications. The experiment with AI in generating abstracts of academic articles highlights the potential of this technology to aid student learning and promote the development of academic research. At the same time, this experiment highlights the potential risks of AI: inaccurate access to information, impeded development of students' critical thinking, and questionable academic integrity and devaluation of academic value.To sum up,AI can be used as a tool of efficiency to assist students and researchers, AI cannot completely replace humans in academic research, AI may not be a substitute for laziness and lack of thought.As AI continues to evolve, people need to find a balance between using AI wisely. In conclusion, it is important for humans to properly control, critically assess, and guide its collaboration with academic practice to ensure that it remains a powerful aid to academic research and not a stumbling block on the path to academic development. Part 2: Write a 600 word summary This article summarizes how the media in four countries - the United States, Australia, India, and Nigeria - portray the existence of climate change and the distribution of responsibility for global climate change by examining media coverage of climate change in these countries. The central thesis of this article is that different media narratives about climate change affect public perceptions of climate change and their views on the allocation of responsibility, as well as the development and implementation of climate-related policies. While the media in the four countries consistently recognize that climate change is a real and urgent environmental problem caused by human activity, they vary in their narratives of responsibility for climate change. The Australian media mainly deflected responsibility from their own country to countries such as India and China, while the media in the United States focused on their own country; the Indian media emphasized that the northern hemisphere was to blame for climate change, while the Nigerian media emphasized that the globe was globally responsible for climate change. The article argues this central thesis through an analysis of media coverage in the four countries, emphasizing that media coverage is nearly unanimous in its scientific consensus on why climate change is occurring, but divergent in its attribution of responsibility for climate change, reflecting uneven political and economic development between countries. The study emphasizes the significant influence of political narratives on media narratives, which sometimes overshadow scientific evidence in shaping public perceptions and policy responses. For example, the Australian media's denial of its responsibility for climate change under the theme of "differential responsibility" contrasts with the self-reflexive and critical stance of the United States media, reflecting the influence of national political contexts on media coverage. This article provides a unique perspective to answer the research readiness question by providing an in-depth analysis of media coverage of climate change in different countries. The article's analysis contributes to our understanding of how the media influences public perceptions of responsibility for climate change and points to the complexity of equitably distributing responsibility in global climate action.This paper expresses insights into the politicization of responsibility for climate change through an examination of media coverage and suggests that a deeper understanding of the complexity of global climate action is essential. It emphasizes the critical importance of communicating the facts of climate change based on existing scientific knowledge and nuanced media narratives, unencumbered by political discourse, to shape public perceptions of climate action and thus drive effective policy implementation. In confronting the research-ready question of the distribution of responsibility for global climate change, this paper illustrates that the media plays a crucial role in constructing public positions of responsibility. It suggests that if the media wants the public to have a fair and objective view of responsibility for climate change, then it needs to confront and move beyond politically tainted narratives to report objectively and clearly on the more scientifically grounded attribution of responsibility for climate change emissions. The narratives of the media in these four countries exemplify the need for the media to adopt a narrative approach that balances political discourse with scientific facts, recognizing historical emissions and per capita differences as a way of fostering a sense of public responsibility for global climate change. Such a change in the media is critical to advocating for fair and effective climate policies, and for countries to take responsibility for mitigating global climate change. This article advocates that media discourse and media coverage should better reflect the complex interplay between historical emissions, stage of economic development, and the distribution of responsibility for global climate change in order to provide a fairer, more equitable, and rational framework of responsibility for international climate negotiations and action planning. In summary, this article not only dissects the current state of media coverage on climate responsibility, but also emphasizes the role and responsibility of media narratives in actions to address global climate change, as well as the centrality of scientific knowledge in shaping public perceptions and policy responses, and provides important recommendations on how to equitably allocate responsibility for climate action, more successfully address climate change, and the impact of the current media landscape on climate action . This leads to the conclusion that the media should accurately and comprehensively report on the shared responsibilities of different countries and groups of people, and use their influence to promote a fairer, objective and scientific understanding of climate change, thereby contributing to the effective implementation of global climate change action. Bibliography Ranjini, Murali, Aishwarya, Kuwar, Harini, Nagendra. “Who’s responsible for climate change? Untangling threads of media discussion in India, Nigeria, Australia, and the USA,”Climate Change, (2021), Vol. 164: 51. https://doi.org/10.1007/s10584-021-03031-1 Crawford, Kate. 2021. Atlas of AI : Power, Politics, and the Planetary Costs of Artificial Intelligence.New Haven,CT: Yale University Press. https://doi.org/10.12987/9780300252392. Jones, Nicola.“AI Visionary.”Nature (London) 624, no. 7992 (2023): 503–503. https://doi.org/10.1038/d41586-023-03925-3. Morocco-Clarke, Ayodele, Fadila Abubakar Sodangi, and Fatima Momodu. “The Implications and Effects of ChatGPT on Academic Scholarship and Authorship: A Death Knell for Original Academic Publications?” Information & Communications Technology Law ahead-of-print, no. ahead-of-print (2023): 1–21. https://doi.org/10.1080/13600834.2023.2239623.
Information Retrieval and Data Mining (COMP0084) Coursework 1 Marking penalties specific to this coursework Marking penalties are denoted with bold font through the specification of the coursework. The maximum amount of marks (= 20) that can be lost due to penalties is the sum of the following: • 5 marks formalformed submission filenames and / or imposing a directory structure, • 5 marks for a report that did not follow the formatting guidelines, • 1 mark for having figures that are not in vector graphics, • 5 marks for incorrectly formed output files (1 mark per output file), and • 4 marks for very slow to execute source code files (1 mark per file). Penalty marks are subtracted from your final mark. To avoid the majority of the aforementioned penalties, please follow the coursework’s guidelines as described below. Task definition An information retrieval model is an essential component for many applications such as search, question answering, and recommendation. In this coursework you are going to develop informa- tion retrieval models that solve the problem of passage retrieval,i.e. given a text query, we need to return a ranked list of short texts (passages). More specifically, in this assignment you are going to build a passage re-ranking system: given a candidate list of returned passages for a text query, you should re-rank these returned passages based on an information retrieval model. qid pid query passage gNorth Central…5274331537731types of dysarthria fr Figure 1: Sample rows from the candidate-passages-top1000 .tsv file. Data Data can be downloaded from this online repository. The data set consists of 3 files: • test-queries.tsv is a tab separated file, where each row contains a test query identifier (qid) and the actual query text. • passage-collection .txt is a collection of passages, one per row. • candidate-passages-top1000 .tsv is a tab separated file with an initial selection of at most 1000 passages for each of the queries in test-queries.tsv. The format of this file is , where pid is the identifier of the passage retrieved, query is the query text, and passage is the passage text (all tab separated). The passages contained in this file are the same as the ones in passage-collection .txt. However, there might be some repetitions, i.e. the same passage could have been returned for more than 1 query. Figure 1 shows some sample rows from that file. It is important that you do not change the above filenames, i.e. your source code must use the above filenames, otherwise we might not be able to assess your code automatically and a penalty of 5 marks will be applied. Coursework tasks and deliverables Please read carefully as all deliverables (D1 to D12) are described within the tasks. If adeliv- erable requests an answer in a specific format, follow these guidelines carefully as parts of the marking will be automated. Although we strongly suggest using Python, you could also use Java. Please make one consistent choice for this coursework. Do not submit your source code as a Jupyter Notebook or similar. All source code must be your own. You are not allowed to reuse any code that is available from someone or somewhere else. Do not use external functions or libraries that can solve (end-to-end) the tasks of building an inverted index, TF-IDF, the vector space model, BM25, or any of the query likelihood language models. You could use functions or libraries that enhance numeric operations, data storage, and retrieval when this does not violate the aforementioned conditions. However, do make sure that your code is not very inefficient. Very inefficient submissions will be penalised. The expected completion time also depends on computing power. As an approximate rule of thumb, the maximum runtime per task should not exceed 30 to 40 minutes when using a modern laptop (e.g. a 2020 MacBook Pro with a first generation M1 processor). The equivalent runtime for very efficient implementations could be 5 minutes or less. If we are not able to timely run your source code on our machines (i.e. the code for a task is still running after about 30 to 40 minutes), a penalty of 1 mark for each task that is slow will be applied. Use only unigram (1-gram) text representations to solve this coursework’s tasks. Write your report using the ACL LATEX template, and submit 1 PDF file named report.pdf. All plots/figures in your report must be in vector graphics format (e.g. PDF). If they are not, a penalty of 1 mark will be applied. Your report should not exceed 6 pages. If there are formatting issues (ACL LATEX template not used, reported exceeds 6 pages etc.) a penalty of 5 marks will be applied. In total, your submission should be made of 4 source code files (and optionally a require- ments .txt file), 5 output CSV files, and 1 PDF file with the report. Do not deploy any internal directory structure in your submission, i.e. just submit the aforementioned 10 or 11 files. Do not compress (e.g. zip) your submission files. The data files should not be included in your submission. You should assume that the data files will be in the same directory as the source code (again no directory structure). If you impose a directory structure, then we will need to remove this by editing your source code to be able to assess your submission automatically. This will result in a 5 mark penalty. Your source code may generate intermediate files when it runs (e.g. to transfer an outcome of a previous step to the next one). These should be removed when submitting your solution. Task 1 – Text statistics (30 marks). Use the data in passage-collection .txt for this task. Extract terms (1-grams) from the raw text. In doing so, you can also perform basic text preprocessing steps. However, you can also choose not to. D1 Do not remove stop words in the first instance. Describe and justify any other text prepro- cessing choice(s), if any, and report the size of the identified indexof terms (vocabulary). Then, implement a function (or a block of source code) that counts the number of occur- rences of terms in the provided data set, plot (Figure 1) their probability of occurrence (normalised frequency) against their frequency ranking, and qualitatively justify that these terms follow Zipf’s law. As a reminder, Zipf’s law for text sets s = 1 in the Zipfian distribution defined by where f(·) denotes the normalised frequency of a term, k denotes the term’s frequency rank in our corpus (with k = 1 being the highest rank), N is the number of terms in our vocabulary, and s is a distribution parameter. How does your empirical distribution compare to the actual Zipf’s law distribution? Use Eq. 1 to explain and to quantify where their difference is coming from, and also compare the two in a log-log plot (Figure 2). How will the difference between the two distributions be affected if we also remove stop words? Justify your answer by depicting (Figure 3) and quantifying this difference. D2 Submit your source code for Task 1 as a single Python or Java file titled task1 .py or task1.java, respectively. Task 2 – Inverted index (15 marks). Use candidate-passages-top1000 .tsv for this task (unique instances of column pairs pid and passage). Using the vocabulary of terms identified in Task 1 (you will need to choose between removing or keeping stop words), build an inverted index for the collection so that you can retrieve passages in an efficient way. D3 Provide a description of your approach to generating an inverted index, report the infor- mation that you decided to store in it, and justify this choice. Hint: Implementing Tasks 3 and 4 should inform this choice. D4 Submit the source code for Task 2 as a single Python or Java file titled task2.py or task2.java, respectively. Task 3 – Retrieval models (35 marks). Use test-queries.tsv and candidate-passages- top1000 .tsv for this task. There is no need to describe your approach in the report. You will be marked based on your output only. If the output does not follow the description (see below), we will apply a penalty of 1 mark per problematic output file and attempt to fix minor issues manually (e.g. remove headers, correct spacing etc.). However, if the output file does not have the exact number of rows as requested (this means that there is a fundamental flow in your solution), you will receive 0 marks for that part of the coursework. We will not be able to re-arrange the order of rows that you have provided (again, please follow the instructions). D5 Extract the TF-IDF vector representations of the passages using the inverted index you have constructed. Using the IDF representation from the corpus of the passages, extract the TF-IDF representation of the queries. Use a basic vector space model with TF-IDF and cosine similarity to retrieve at most 100 passages from within the 1000 candidate passages for each query (some queries have fewer candidate passages). Store the outcomes in a file named tfidf .csv. Each row of tfidf .csv must have the following format: qid,pid,score where qid denotes the query’s identifier, pid denotes the passage identifier, and score is their cosine similarity score. Note that there should be no spaces between commas. The output .csv files should not have headers. Strictly report the identifiers from the raw data (do not replace them with your own). qid,pid pairs should be ranked by similarity score in descending order,i.e. starting with the highest similarity score. You should first report the top qid,pid pairs for one qid, then move on to the next, following the same query order as in the text-queries.tsv file. The same file format should be used for reporting the outputs of all the retrieval or language models that you will implement as part of this coursework (i.e. this applies to Task 4 as well). The last column should report the corresponding similarity/relevance score of each model. Each output .csv file is expected to have exactly 19,290 rows. D6 Use the inverted index to also implement BM25 while setting k1 = 1.2, k2 = 100, and b = 0.75. Retrieve at most 100 passages from within the 1000 candidate passages for each test query. Store the outcomes in a file named bm25 .csv. D7 Submit the source code for Task 3 as a single Python or Java file titled task3.py or task3.java, respectively. Task 4 – Query likelihood language models (20 marks). Use test-queries.tsv and candidate-passages-top1000 .tsv for this task. Implement the query likelihood language model with (a) Laplace smoothing, (b) Lidstone correction with ϵ = 0.1, and (c) Dirichlet smoothing with µ = 50, and retrieve at most 100 passages from within the 1000 candidate pas- sages for each test query. There is no need to describe your approach in implementing these language models in the report; you will be marked based on your output only. However, please note that D11 needs to be answered in the report. Similarly to Task 3, if the output does not follow the description (see Task 3), we will apply a penalty of 1 mark per problematic output file and attempt to fix minor issues manually (e.g. remove headers, correct spacing etc.). However, if the output file does not have the exact number of rows as requested (this means that there is a fundamental flow in your solution), you will receive 0 marks for that part of the coursework. We will not be able to re-arrange the order of rows that you have provided (again, please follow the instructions). D8-10 Store the respective outcomes in the files laplace.csv, lidstone.csv, and dirichlet .csv. In these files, the column score should report the natural logarithm of the probability scores. D11 Which language model do you expect to work better? Give a few empirical examples based on your data and experiments. The examples need to be comprehensive to a reader of the report that does not have access to the data (i.e. do not refer to passage or query IDs). Which language models are expected to be more similar and why? Give a few empirical examples based on your data and experiments. Comment on the value of ϵ = 0.1 in the Lidstone correction. Is this a good choice (yes/no), would there be a better setting (if so, please provide a range of values), and why? If we set μ = 5000 in Dirichlet smoothing, would this be a more appropriate value (yes/no), and why? D12 Submit the source code for Task 4 as a single Python or Java file titled task4.py or task4.java, respectively.
MCCA104 – Engaging Media Assessment Guide Trimester 1A, 2024 UNIT OUTLINE AND ASSESSMENT GUIDE The Unit Outline and the Assessment Guide for Engaging Media provide the full requirements of the unit and both should be referred to when completing unit assessments. PRESENTATION OF WRITTEN ASSIGNMENTS In all the written assessments for this unit, it is important that you present your work in a way that communicates a professional attitude. Below are some formatting rules that should be followed for each written assessment : • Type or word-process your assessment • Use a 12 font size • Use Arial font • Leave a margin of 2.5 cm around all four sides of the page • Use double spacing between lines • Use a Header to insert you name and student ID number at the top of the page • Number pages clearly • Keep a copy of your submitted assessments For Reference List: • Formatted according the APA 7th Edition referencing system; • Use single space for each source entry; and • Use double space between entries. • Use hanging indentation (from the second line of each reference) • Alphabetise the reference list by author name DETAILED INFORMATION ON ASSESSMENT TASKS Assessment 1: Learning portfolio Due Date: Sunday Week 6, 11.55pm via Turnitin Weighting: 30% Assessment Task The learning portfolio is like an ongoing learning ‘diary’ that you maintain in detail for the first 6 weeks of the unit. It aims to guide you through the tasks and skills you require for completing the other assessment tasks in the unit. This assessment will assist you in achieving the unit outcomes by documenting your: - ability to find and evaluate appropriate sources of information - ability to critically analyse texts and develop critical thinking skills - ability to synthesise ideas and effectively communicate in a formal manner - ability to identify arguments and counter-arguments Submission Requirements Each week there will be specific tasks you need to carry out and record in your learning portfolio. You will need to document your portfolio in a Word Document, using the presentation guides (formatting) as outlined on page 2 of this document. Each week you will be required to show that you have completed each weekly task and this will be marked off on a log, this is to help guide your time- management of this assessment and is part of your final assessment mark. The tasks and writing exercises build on each other to ensure that you have the required skills and experience to tackle the next two assessments in the unit. You will need to submit all of the logs at the end of week 6 one Word document via Turnitin. (The submission point is under the week 6 tab on Moodle). Please save this document using the following pattern: MCCA104_A.1_(insert your full name) Assessment 2: Remediation Project Due Date: Sunday Week 9, 11.55pm via Turnitin Weighting: 30% Assessment Task Submission Requirements This assessment consists of two parts. A practical creative response to the notion of remediation (remediation component), as well as research demonstrating your understanding of the processes of production, distribution and consumption, and how these may be changing in a digital media environment (written component). This assessment assists you in achieving the unit learning outcomes because you will demonstrate: - your practical skills in, and engagement with, the concept of ‘remediation’ and the contemporary media environment - your critical understanding of the various factors influencing the production and consumption of media texts - your ability to present these understandings in both written and creative form using scholarly references Remediation Component: Choose a media text that you enjoy and think would make a good candidate for ‘remediation’ (i.e. representing it in a different media form). Your text could come from any form of mass media. Some good examples include (but are not limited to): - A book - A poem - A song - A video - A newspaper article - A photograph - A speech You need to remediate this text – i.e. re-present the text in a different media format. For example, if the original text was a newspaper article, you could rework it as a webpage, or a video. You do not have to stick to a strict definition of what a remediation is; your new text may involve re-mixing, mash- ups and changing the original context/text, but you’ll have to think about and justify why you have a more open approach to remediation in the written component. Your remediated text needs to be presented in an electronic format (e.g., as a pdf, video, webpage, blog, jpeg, etc.) As a general guide, audio or video texts should be no more than about 3 minutes long; involved graphical texts (e.g. graphic novel, cartoons) should be no more than 12 panels; text based remediations (including websites, blogs, newspaper article etc.) should be no more than 1100 words. It is advised that you choose from non-copyrighted materials for both your text and when creating your remediation. These can be found by selecting ‘tools’ on a google search and then filtering your searches to ‘labelled for reuse with modification’ . (Please see screenshot below.) Your remediation will not be assessed according to technological or artistic prowess, rather, on the creative use of the platforms affordances and on the success of the process of transforming the text into a different form and the understanding it suggests of the shifts in production, distribution and/or audience in a new/digital media environment. (Hint: it will be hard t demonstrate this understanding if your new text does not differ much in terms of production, distribution or consumption from the old text). In order to fully engage with the issues surrounding new media production, you are required to upload your Remediation online to a platform appropriate to your chosen type of remediation. If possible, this upload should be kept private so that only you can see it. Once you have uploaded this remediation, you will need to screenshot this and email your screenshots and original remediation file to your teacher. (Their email address can be found in the Unit Outline.) The files you have uploaded should then be deleted. Written Component: Along with the remediation you need to submit a short-written document that outlines your understanding of how your remediation changes the way in which the original text can/has been produced, distributed and consumed. Your documentation should take the following format: Original Text (200 words +/- 10% you only need to briefly list these) - Title - Format - Producers (list in as much detail as possible. E.g. the people, companies or institutions involved in the production of the text). - Distributors (list in as much detail as possible about the people, companies or institutions involved in the distribution of the text.) - Consumers (identify the intended and/or likely audiences for this text.) Remediated Text (200 words +/- 10% you only need to briefly list these) - Title - Format - Production (who/what was – or could – be involved in the production of the text) - Distribution (how might this text be distributed? Who would be involved?) - Audience (identify the intended and / or likely audiences for this text.) Provide a response to the following (1000 words +/- 10%): 1. In your own words, briefly describe the process of remediating your text and articulate what might be differences in meaning between the original and remediated text/s. 2. In what ways does your remediated text demonstrate an understanding of how remediation impacts on the contemporary media environment? 3. What key themes and/or concepts covered in the unit do you think are raised by your remediation? Why/ How? Assessment 3: Essay Due Date: Sunday Week 12, 11.55pm via Turnitin Weighting: 40% Assessment Task Submission Requirements Write a 1500-word essay responding to one of the questions below. Each question roughly relates to one of the key themes or issues considered throughout the unit. 1. Traditional applications of copyright are becoming unworkable in the contemporary media environment. Critically discuss, utilising the concepts you have learned in the unit and applying critical understandings of media cultures andinstitutions. 2. Remediation is a key contemporary practise that performs McLuhan’s earlier notion that ‘the medium is the message.’ Critically discuss, utilising the concepts you have learned in the unit and applying critical understandings of media cultures andinstitutions. 3. Digital culture has changed the way we produce, distribute and consume music. Critically discuss, utilising the concepts you have learned in the unit and applying critical understandings of media cultures andinstitutions. 4. Globalisation encourages the production of culturally diverse media. Critically discuss, utilising the concepts you have learned in the unit and applying critical understandings of media cultures andinstitutions. 5. Digital culture has changed the way we produce, distribute and consume news media. Critically discuss, utilising the concepts you have learned in the unit and applying critical understandings of media cultures andinstitutions. 6. Privacy is becoming increasingly relevant, and yet at risk in the context of social networking sites and social media. Critically discuss, utilising the concepts you have learned in the unit and applying critical understandings of media cultures andinstitutions. 7. In a period of ‘post-truth’ media, the notion of credibility is becoming increasingly relevant, and yet difficult to obtain and discern. Critically discuss, utilising the concepts you have learned in the unit and applying critical understandings of media cultures andinstitutions. 8. The success of digital games has changed the way we think about other traditional media production and consumption. Critically discuss, utilising the concepts you have learned in the unit and applying critical understandings of media cultures andinstitutions. 9. The performance of identity online is increasingly intertwined with our offline identity. Critically discuss, utilising the concepts you have learned in the unit and applying critical understandings of media cultures.
Strategy 2 for MAM – Final Exam 2024 Spring Based on your reading of the “Apple’s iPhone Revolution: Pioneering the Circular Economy” case (9-123-089), please answer the following questions. Make sure to clearly indicate the beginning and end of the answer to each question. The exam counts toward 50 percent of your grade. Note: Please quantify your arguments using relevant data from the case and your assumptions. Assignment Questions: (1) What are the opportunities and risks from the Trade-In system for Apple? Please think from both financial performance and environmental impact perspectives and provide qualitative analysis. (20 points) (2) What are the revenue and operating profitability impacts from the second-hand market for iPhones for Apple in 2021? (16 points). Please provide quantitative analysis. Assume: · Net price of refurbished iPhone sold by Apple in 2022: $600 · Second-hand iPhone users 50% as likely to spend on services · Percentage of iPhone buyers in first-hand market that buy again an iPhone: 80% You can make other assumptions if needed. (3) How many carbon emissions have been avoided from the second-hand market for iPhones in 2022? Please provide quantitative analysis and specify your assumptions. You may consider optimistic and pessimistic scenarios (14 points). If the question is not clear enough, state your assumptions, and if they are reasonable, you will be given credit. This is an INDIVIDUAL exam. You ARE NOT permitted to consult with ANYONE about the exam. You ARE NOT permitted to use any material other than the cases, the readings in the course pack, your class notes, and slides distributed in class. The exam should be no more than 3 pages (including exhibits). It should be typed larger than or equal to 10pt Times New Roman font, with a 1-inch margin on all four sides. The exam should be submitted on Canvas by 7th April 2024. The window for submissions will close at 6pm - no exceptions. This ensures that all students get the same time to complete the exam. You can, of course, submit the exam anytime earlier.
Assignment I (Semester B, 2024/2025) CS5293 Topics on Information Security Due: Saturday 22nd February, 2025; 100 Marks 1 Introduction This is the 1st Assignment for the course CS5293 in the semester B of academic year 2024-2025. 1.1 Objective The learning objective of this assignment is for you to get a deeper understanding on several fundamental cryptographic building blocks and some common vulnerabilities in general software. After finishing the assignment, you should be able to gain a first-hand experience on symmetric encryption algorithms, en-cryption modes, paddings, initial vector (IV), one-way hash functions, message authentication code (MAC), digital signature, pseudorandomness generator (PRNG), etc. 1.2 Environment All tasks in this assignment can be done on the VirtualBox as introduced in Tutorials. You can also find a manual named tut1-Environment-Setup.pdf on the Canvas. You may even check http://www.cis.syr.edu/~wedu/seed/lab_env.html to download useful materials such as set up manuals and OS images to establish the environment. Some helpful document about OpenSSL Command-Line HOWTO" can be found at http://www.madboa.com/geek/openssl/. 1.3 Due Date The assignment is due on Saturday 22nd February, 2025 (week 5). Any reports should be submitted before 23:59:59 (Firm!). 1.4 Submission You will submit a report to describe what you have done and what you have observed with screen shots whenever necessary; you also need to provide explanation or codes to the observations that are interesting or surprising. In your report, you need to answer all the questions listed in this manual. Please answer each question using at most 100 words. Your report will be written methodically and clearly in order with index and all chapters presented. Typeset your report into .pdf file (make sure it can be opened with Adobe Reader) and name it as the format: [Your Name]-[Student ID]-CS5293-Assignment1.pdf, e.g., HarryPotter-12345678-CS5293-Assignment1.pdf. Then, upload the PDF file to Canvas. 2 Secret-Key Encryption [35 Marks] The learning objective of this exercise is for you to get familiar with the concepts in the secret-key encryption. After finishing the tasks, you should be able to gain a first-hand experience on encryption algorithms, encryption modes, paddings, and initial vector (IV). Moreover, you will be able to use tools and write programs to encrypt/decrypt messages. 2.1 Task 1: Frequency Analysis Against Monoalphabetic Substitution Cipher [5 Marks] It is well-known that monoalphabetic substitution cipher (also known as monoalphabetic cipher) is not secure, because it can be subjected to frequency analysis. In this task, you are given a cipher-text that is encrypted using a monoalphabetic cipher; namely, each letter in the original text is replaced by another letter, where the replacement does not vary (i.e., a letter is always replaced by the same letter during the encryption). Your job is to find out the original text using frequency analysis. It is known that the original text is an English article. In the following, we describe how we encrypt the original article, and what simplification we have made. • Step 1: let us do some simplification to the original article. We convert all upper cases to lower cases, and then removed all the punctuations and numbers. We do keep the spaces between words, so you can still see the boundaries of the words in the ciphertext. In real encryption using monoalphabetic cipher, spaces will be removed. We keep the spaces to simplify the task. We did this using the following command: $ tr [:upper:] [:lower:] < article.txt > lowercase.txt $ tr -cd '[a-z][ ][:space:]' < lowercase.txt > plaintext.txt • Step 2: let us generate the encryption key, i.e., the substitution table. We will permute the alphabet from a to z using Python, and use the permuted alphabet as the key. See the following program. $ python >>> import random >>> s = "abcdefghijklmnopqrstuvwxyz" >>> list = random.sample(s, len(s)) >>> ''.join(list) 'sxtrwinqbedpvgkfmalhyuojzc' • Step 3: we use the tr command to do the encryption. We only encrypt letters, while leaving the space and return characters alone. $ tr 'abcdefghijklmnopqrstuvwxyz' 'sxtrwinqbedpvgkfmalhyuojzc' < plaintext.txt > ciphertext.txt We have created a ciphertext using a different encryption key (not the one described above). You can download it from the Canvas. Your job is to use the frequency analysis to figure out the encryption key and the original plaintext. Guidelines. Using the frequency analysis, you can find out the plaintext for some of the characters quite easily. For those characters, you may want to change them back to its plaintext, as you may be able to get more clues. It is better to use capital letters for plaintext, so for the same letter, we know which is plaintext and which is ciphertext. You can use the tr command to do this. For example, in the following, we replace letters a, e, and t in in.txt with letters X, G, E, respectively; the results are saved in out.txt. $ tr 'aet' 'XGE' < in.txt > out.txt There are many online resources that you can use. We list four useful links in the following: • http://www.richkni.co.uk/php/crypta/freq.php: This website can produce the statistics fro a ciphertext, including the single-letter frequencies, bigram frequencies (2-letter sequence), and trigram frequencies (3-letter sequence), etc. • https://en.wikipedia.org/wiki/Frequency_analysis: This Wikipedia page provides frequencies for a typical English plaintext. • https://en.wikipedia.org/wiki/Bigram: Bigram frequency. • https://en.wikipedia.org/wiki/Trigram: Trigram frequency. 2.2 Task 2: Encryption using Different Ciphers and Modes [5 Marks] In this task, we will play with various encryption algorithms and modes. You can use the following openssl enc command to encrypt/decrypt a file. To see the manuals, you can type man openssl and man enc. $ openssl enc -ciphertype -e -in plain.txt -out cipher.bin -K 00112233445566778889aabbccddeeff -iv 0102030405060708 Please replace the ciphertype with a specific cipher type, such as -aes-128-cbc, -bf-cbc, -aes-128-cfb, etc. In this task, you should try at least 3 different ciphers. You can find the meaning of the command-line options and all the supported cipher types by typing ”man enc”. We include some common options for the openssl enc command in the following: -in input file -out output file -e encrypt -d decrypt -K/-iv key/iv in hex is the next argument -[pP] print the iv/key (then exit if -P) 2.3 Task 3: Encryption Mode – ECB vs. CBC [5 Marks] The file pic original.bmp can be downloaded from the Canvas, and it contains a simple picture. We would like to encrypt this picture, so people without the encryption keys cannot know what is in the picture. Please encrypt the file using the ECB (Electronic Code Book) and CBC (Cipher Block Chaining) modes, and then do the following: • 1. Let us treat the encrypted picture as a picture, and use a picture viewing software to display it. However, For the .bmp file, the first 54 bytes contain the header information about the picture, we have to set it correctly, so the encrypted file can be treated as a legitimate .bmp file. We will replace the header of the encrypted picture with that of the original picture. We can use the bless hex editor tool (already installed on our VM) to directly modify binary files. We can also use the following commands to get the header from p1.bmp, the data from p2.bmp (from offset 55 to the end of the file), and then combine the header and data together into a new file. $ head -c 54 p1.bmp > header $ tail -c +55 p2.bmp > body $ cat header body > new.bmp • 2. Display the encrypted picture using a picture viewing program (we have installed an image viewer program called eog on our VM). Can you derive any useful information about the original picture from the encrypted picture? Please explain your observations. Select a picture of your choice, repeat the experiment above, and report your observations. 2.4 Task 4: Padding [5 Marks] For block ciphers, when the size of a plaintext is not a multiple of the block size, padding may be required. All the block ciphers normally use PKCS#5 padding, which is known as standard block padding. We will conduct the following experiments to understand how this type of padding works: 1. Use ECB, CBC, CFB, and OFB modes to encrypt a file (you can pick any cipher). Please report which modes have paddings and which ones do not. For those that do not need paddings, please explain why. 2. Let us create three files, which contain 5 bytes, 10 bytes, and 16 bytes, respectively. We can use the following “echo -n” command to create such files. The following example creates a file f1.txt with length 5 (without the -n option, the length will be 6, because a newline character will be added by echo): $ echo -n "12345" > f1.txt We then use “openssl enc -aes-128-cbc -e” to encrypt these three files using 128-bit AES with CBC mode. Please describe the size of the encrypted files. We would like to see what is added to the padding during the encryption. To achieve this goal, we will decrypt these files using “openssl enc -aes-128-cbc -d”. Unfortunately, decryption by default will automatically remove the padding, making it impossible for us to see the padding. However, the command does have an option called “-nopad”, which disables the padding, i.e., during the decryption, the command will not remove the padded data. Therefore, by looking at the decrypted data, we can see what data are used in the padding. Please use this technique to figure out what paddings are added to the three files. It should be noted that padding data may not be printable, so you need to use a hex tool to display the content. The following example shows how to display a file in the hex format: $ hexdump -C p1.txt 00000000 31 32 33 34 35 36 37 38 39 49 4a 4b 4c 0a |123456789IJKL.| $ xxd p1.txt 00000000: 3132 3334 3536 3738 3949 4a4b 4c0a 123456789IJKL. 2.5 Task 5: Error Propagation – Corrupted Cipher Text [5 Marks] To understand the error propagation property of various encryption modes, we would like to do the following exercise: 1. Create a text file that is at least 1000 bytes long. 2. Encrypt the file using the AES-128 cipher. 3. Unfortunately, a single bit of the 55th byte in the encrypted file got corrupted. You can achieve this corruption using the bless hex editor. 4. Decrypt the corrupted ciphertext file using the correct key and IV Please answer the following question: How much information can you recover by decrypting the corrupted file, if the encryption mode is ECB, CBC, CFB, or OFB, respectively? Please answer this question before you conduct this task, and then find out whether your answer is correct or wrong after you finish this task. Please provide justification. 2.6 Task 6: Initial Vector (IV) [5 Marks] Most of the encryption modes require an initial vector (IV). Properties of an IV depend on the cryptographic scheme used. If we are not careful in selecting IVs, the data encrypted by us may not be secure at all, even though we are using a secure encryption algorithm and mode. The objective of this task is to help you understand the problems if an IV is not selected properly. Please do the following experiments: • Task 6.1. A basic requirement for IV is uniqueness, which means that no IV may be reused under the same key. To understand why, please encrypt the same plaintext using (1) two different IVs, and (2) the same IV. Please describe your observation, based on which, explain why IV needs to be unique. • Task 6.2. One may argue that if the plaintext does not repeat, using the same IV is safe. Let us look at the Output Feedback (OFB) mode. Assume that the attacker gets hold of a plaintext (P1) and a ciphertext (C1), can he/she decrypt other encrypted messages if the IV is always the same? You are given the following information, please try to figure out the actual content of P2 based on C2, P1, and C1. Plaintext (P1): This is a known message! Ciphertext (C1): a469b1c502c1cab966965e50425438e1bb1b5f9037a4c159 Plaintext (P2): (unknown to you) Ciphertext (C2): bf73bcd3509299d566c35b5d450337e1bb175f903fafc159 If we replace OFB in this experiment with CFB (Cipher Feedback), how much of P2 can be revealed? You only need to answer the question; there is no need to demonstrate that. The attack used in this experiment is called the known-plaintext attack, which is an attack model for cryptanalysis where the attacker has access to both the plaintext and its encrypted version (ciphertext). If this can lead to the revealing of further secret information, the encryption scheme is not considered as secure. • Task 6.3. From the previous tasks, we now know that IVs cannot repeat. Another important requirement on IV is that IVs need to be unpredictable for many schemes, i.e., IVs need to be randomly generated. In this task, we will see what is going to happen if IVs are predictable. Assume that Bob just sent out an encrypted message, and Eve knows that its content is either Yes or No; Eve can see the ciphertext and the IV used to encrypt the message, but since the encryption algorithm AES is quite strong, Eve has no idea what the actual content is. However, since Bob uses predictable IVs, Eve knows exactly what IV Bob is going to use next. The following summarizes what Bob and Eve know: Encryption method: 128-bit AES with CBC mode. Key (in hex): 00112233445566778899aabbccddeeff (known only to Bob) Ciphertext (C1): bef65565572ccee2a9f9553154ed9498 (known to both) IV used on P1 (known to both) (in ascii): 1234567890123456 (in hex) : 31323334353637383930313233343536 Next IV (known to both) (in ascii): 1234567890123457 (in hex) : 31323334353637383930313233343537 A good cipher should not only tolerate the known-plaintext attack described previously, it should also tolerate the chosen-plaintext attack, which is an attack model for cryptanalysis where the attacker can obtain the ciphertext for an arbitrary plaintext. Since AES is a strong cipher that can tolerate the chosen-plaintext attack, Bob does not mind encrypting any plaintext given by Eve; he does use a different IV for each plaintext, but unfortunately, the IVs he generates are not random, and they can always be predictable. Your job is to construct a message P2 and ask Bob to encrypt it and give you the ciphertext. Your objective is to use this opportunity to figure out whether the actual content of P1 is Yes or No. You can read this Wikipedia page for ideas: https://en.wikipedia.org/wiki/Initialization_vector. There are more advanced cryptanalysis on IV that is beyond the scope of this task. You can read the article posted in this URL: https://defuse.ca/cbcmodeiv.htm. Because the requirements on IV really depend on cryptographic schemes, it is hard to remember what properties should be maintained when we select an IV. However, we will be safe if we always use a new IV for each encryption, and the new IV needs to be generated using a good pseudo random number generator, so it is unpredictable by adversaries. (See tasks about Random Number Generation) for details on how to generate cryptographically strong pseudo random numbers. 2.7 Task 7: Programming using the Crypto Library [5 Marks] In this task, you are given a plaintext and a ciphertext, and your job is to find the key that is used for the encryption. You do know the following facts: • The aes-128-cbc cipher is used for the encryption. • The key used to encrypt this plaintext is an English word shorter than 16 characters; the word can be found from a typical English dictionary. Since the word has less than 16 characters (i.e. 128 bits), pound signs (#: hexadecimal value is 0x23) are appended to the end of the word to form. a key of 128 bits. Your goal is to write a program to find out the encryption key. You can download a English word list from the Internet. We have also linked one on the (Canvas). The plaintext, ciphertext, and IV are listed in the following: Plaintext (total 21 characters): This is a top secret. Ciphertext (in hex format): 764aa26b55a4da654df6b19e4bce00f4 ed05e09346fb0e762583cb7da2ac93a2 IV (in hex format): aabbccddeeff00998877665544332211 You need to pay attention to the following issues: • If you choose to store the plaintext message in a file, and feed the file to your program, you need to check whether the file length is 21. If you type the message in a text editor, you need to be aware that some editors may add a special character to the end of the file. The easiest way to store the message in a file is to use the following command (the -n flag tells echo not to add a trailing newline): $ echo -n "This is a top secret." > file • In this task, you are supposed to write your own program to invoke the crypto library. No credit will be given if you simply use the openssl commands to do this task. Sample code can be found from the following URL: https://www.openssl.org/docs/man1.0.2/crypto/EVP_EncryptInit.html • When you compile your code using gcc, do not forget to include the -lcrypto flag, because your code needs the crypto library. See the following example: $ gcc -o myenc myenc.c -lcrypto 3 MD5 Collision Attack [20 Marks] The learning objective of this exercise is for you to really understand the impact of collision attacks, and see in first hand what damages can be caused if a widely-used one-way hash function’s collision-resistance property is broken. To achieve this goal, you need to launch actual collision attacks against the MD5 hash function. Using the attacks, you should be able to create two different programs that share the same MD5 hash but have completely different behaviors. The tasks use a tool called Fast MD5 Collision Generation, which was written by Marc Stevens; the name of the binary is called md5collgen in our VMs. For Ubuntu16.04 VM: md5collgen has already been installed inside /home/seed/bin. 3.1 Task 8: Generating Two Different Files with the Same MD5 Hash [5 Marks] In this task, we will generate two different files with the same MD5 hash values. The beginning parts of these two files need to be the same, i.e., they share the same prefix. We can achieve this using the md5collgen program, which allows us to provide a prefix file with any arbitrary content. The way how the program works is illustrated in the figure following. The following command generates two output files, out1.bin and out2.bin, for a given a prefix file prefix.txt: $ md5collgen -p prefix.txt -o out1.bin out2.bin We can check whether the output files are distinct or not using the diff command. We can also use the md5sum command to check the MD5 hash of each output file. See the following commands. $ diff out1.bin out2.bin $ md5sum out1.bin $ md5sum out2.bin Since out1.bin and out2.bin are binary, we cannot view them using a text-viewer program, such as cat or more; we need to use a binary editor to view (and edit) them. We have already installed a hex editor software called bless in our VM. Please use such an editor to view these two output files, and describe your observations. In addition, you should answer the following questions. • Question 1. If the length of your prefix file is not multiple of 64, what is going to happen? • Question 2. Create a prefix file with exactly 64 bytes, and run the collision tool again, and see what happens. • Question 3. Are the data (128 bytes) generated by md5collgen completely different for the two output files? Please identify all the bytes that are different. 3.2 Task 9: Understanding MD5’s Property [5 Marks] In this task, we will try to understand some of the properties of the MD5 algorithm. These properties are important for us to conduct further tasks. MD5 is a quite complicated algorithm, but from very high level, it is not so complicated. As Figure 2 shows, MD5 divides the input data into blocks of 64 bytes, and then computes the hash iteratively on these blocks. The core of the MD5 algorithm is a compression function, which takes two inputs, a 64-byte data block and the outcome of the previous iteration. The compression function produces a 128-bit IHV, which stands for “Intermediate Hash Value”; this output is then fed into the next iteration. If the current iteration is the last one, the IHV will be the final hash value. The IHV input for the first iteration (IHV0) is a fixed value. Based on how MD5 works, we can derive the following property of the MD5 algorithm: Given two inputs M and N, if MD5(M) = MD5(N), i.e., the MD5 hashes of M and N are the same, then for any input T, MD5(M || T) = MD5(N || T), where || represents concatenation. That is, if inputs M and N have the same hash, adding the same suffix T to them will result in two outputs that have the same hash value. This property holds not only for the MD5 hash algorithm, but also for many other hash algorithms. Your job in this task is to design an experiment to demonstrates that this property holds for MD5. You can use the cat command to concatenate two files (binary or text files) into one. The following command concatenates the contents of file2 to the contents of file1, and places the result in file3. $ cat file1 file2 > file3 3.3 Task 10: Generating Two Executable Files with the Same MD5 Hash [5 Marks] In this task, you are given the following C program. Your job is to create two different versions of this program, such that the contents of their xyz arrays are different, but the hash values of the executables are the same. #include unsigned char xyz[200] = { /* The actual contents of this array are up to you */ }; int main() { int i; for (i=0; i prefix $ tail -c 100 a.out > suffix $ tail -c +3300 a.out > suffix The first command above saves the first 3200 bytes of a.out to prefix. The second command saves the last 100 bytes of a.out to suffix. The third command saves the data from the 3300th byte to the end of the file a.out to suffix. With these two commands, we can divide a binary file into pieces from any location. If we need to glue some pieces together, we can use the cat command. If you use bless to copy-and-paste a block of data from one binary file to another file, the menu item ”Edit → Select Range” is quite handy, because you can select a block of data using a starting point and a range, instead of manually counting how many bytes are selected. 3.4 Task 11: Making the Two Programs Behave Differently [5 Marks] In the previous task, we have successfully created two programs that have the same MD5 hash, but their behaviors are different. However, their differences are only in the data they print out; they still execute the same sequence of instructions. In this task, we would like to achieve something more significant and more meaningful. Assume that you have created a software which does good things. You send the software to a trusted authority to get certified. The authority conducts a comprehensive testing of your software, and concludes that your software is indeed doing good things. The authority will present you with a certificate, stating that your program is good. To prevent you from changing your program after getting the certificate, the MD5 hash value of your program is also included in the certificate; the certificate is signed by the authority, so you cannot change anything on the certificate or your program without rendering the signature invalid. You would like to get your malicious software certified by the authority, but you have zero chance to achieve that goal if you simply send your malicious software to the authority. However, you have noticed that the authority uses MD5 to generate the hash value. You got an idea. You plan to prepare two different programs. One program will always execute benign instructions and do good things, while the other program will execute malicious instructions and cause damages. You find a way to get these two programs to share the same MD5 hash value. You then send the benign version to the authority for certification. Since this version does good things, it will pass the certification, and you will get a certificate that contains the hash value of your benign program. Because your malicious program has the same hash value, this certificate is also valid for your malicious program. Therefore, you have successfully obtained a valid certificate for your malicious program. If other people trusted the certificate issued by the authority, they will download your malicious program. The objective of this task is to launch the attack described above. Namely, you need to create two programs that share the same MD5 hash. However, one program will always execute benign instructions, while the other program will execute malicious instructions. In your work, what benign/malicious instruc-tions are executed is not important; it is sufficient to demonstrate that the instructions executed by these two programs are different. Guidelines. Creating two completely different programs that produce the same MD5 hash value is quite hard. The two hash-colliding programs produced by md5collgen need to share the same prefix; moreover, as we can see from the previous task, if we need to add some meaningful suffix to the outputs produced by md5collgen, the suffix added to both programs also needs to be the same. These are the limitations of the MD5 collision generation program that we use. Although there are other more complicated and more advanced tools that can lift some of the limitations, such as accepting two different prefixes [see here], they demand much more computing power, so they are out of the scope for this task. We need to find a way to generate two different programs within the limitations. There are many ways to achieve the above goal. We provide one approach as a reference, but you are encouraged to come up their own ideas. In our approach, we create two arrays X and Y. We compare the contents of these two arrays; if they are the same, the benign code is executed; otherwise, the malicious code is executed. See the following pseudo-code: Array X; Array Y; main() { if(X's contents and Y's contents are the same) run benign code; else run malicious code; return; } We can initialize the arrays X and Y with some values that can help us find their locations in the executable binary file. Our job is to change the contents of these two arrays, so we can generate two different versions that have the same MD5 hash. In one version, the contents of X and Y are the same, so the benign code is executed; in the other version, the contents of X and Y are different, so the malicious code is executed. We can achieve this goal using a technique similar to the one used in Task 3. Figure 4 illustrates what the two versions of the program look like. From Figure 4, we know that these two binary files have the same MD5 hash value, as long as P and Q are generated accordingly. In the first version, we make the contents of arrays X and Y the same, while in the second version, we make their contents different. Therefore, the only thing we need to change is the contents of these two arrays, and there is no need to change the logic of the programs. 4 RSA Public-Key Encryption and Signature [24 Marks] The learning objective is for you to gain hands-on experiences on the RSA algorithm. From lectures, you should have learned the theoretic part of the RSA algorithm, so you know mathematically how to generate public/private keys and how to perform. encryption/decryption and signature generation/verification. The tasks enhance your understanding of RSA by requiring you to go through every essential step of the RSA algorithm on actual numbers, so they can apply the theories learned from the class. Essentially, you will be implementing the RSA algorithm using the C program language. Description. The RSA algorithm involves computations on large numbers. These computations cannot be directly conducted using simple arithmetic operators in programs, because those operators can only operate on primitive data types, such as 32-bit integer and 64-bit long integer types. The numbers involved in the RSA algorithms are typically more than 512 bits long. For example, to multiple two 32-bit integer numbers a and b, we just need to use a*b in our program. However, if they are big numbers, we cannot do that any more; instead, we need to use an algorithm (i.e., a function) to compute their products. There are several libraries that can perform. arithmetic operations on integers of arbitrary size. For the tasks, we will use the Big Number library provided by openssl. To use this library, we will define each big number as a BIGNUM type, and then use the APIs provided by the library for various operations, such as addition, multiplication, exponentiation, modular operations, etc. 4.1 BIGNUM APIs All the big number APIs can be found from https://linux.die.net/man/3/bn. In the following, we describe some of the APIs that are needed for the tasks. • Some of the library functions requires temporary variables. Since dynamic memory allocation to create BIGNUMs is quite expensive when used in conjunction with repeated subroutine calls, a BN CTX structure is created to holds BIGNUM temporary variables used by library functions. We need to create such a structure, and pass it to the functions that requires it. BN_CTX *ctx = BN_CTX_new() • Initialize a BIGNUM variable BIGNUM *a = BN_new() • There are a number of ways to assign a value to a BIGNUM variable // Assign a value from a decimal number string BN_dec2bn(&a, "12345678901112231223"); // Assign a value from a hex number string BN_hex2bn(&a, "2A3B4C55FF77889AED3F"); // Generate a random number of 128 bits BN_rand(a, 128, 0, 0); // Generate a random prime number of 128 bits BN_generate_prime_ex(a, 128, 1, NULL, NULL, NULL); • Print out a big number. void printBN(char *msg, BIGNUM * a) { // Convert the BIGNUM to number string char * number_str = BN_bn2dec(a); // Print out the number string printf("%s %s ", msg, number_str); // Free the dynamically allocated memory OPENSSL_free(number_str); } • Compute res = a − b and res = a + b: BN_sub(res, a, b); BN_add(res, a, b); • Compute res = a ∗ b. It should be noted that a BN CTX structure is need in this API. BN_mul(res, a, b, ctx); • Compute res = a · b mod n: BN_mod_mul(res, a, b, n, ctx); • Compute res = a c mod n: BN_mod_exp(res, a, c, n, ctx); • Compute modular inverse, i.e., given a, find b, such that a · b mod n = 1. The value b is called the inverse of a, with respect to modular n. BN_mod_inverse(b, a, n, ctx); 4.2 A Complete Example We show a complete example in the following. In this example, we initialize three BIGNUM variables, a, b, and n; we then compute a · b and (a b mod n). /* bn_sample.c */ #include #include #define NBITS 256 void printBN(char *msg, BIGNUM * a) { /* Use BN_bn2hex(a) for hex string * Use BN_bn2dec(a) for decimal string */ char * number_str = BN_bn2hex(a); printf("%s %s ", msg, number_str); OPENSSL_free(number_str); } int main () { BN_CTX *ctx = BN_CTX_new(); BIGNUM *a = BN_new(); BIGNUM *b = BN_new(); BIGNUM *n = BN_new(); BIGNUM *res = BN_new(); // Initialize a, b, n BN_generate_prime_ex(a, NBITS, 1, NULL, NULL, NULL); BN_dec2bn(&b, "273489463796838501848592769467194369268"); BN_rand(n, NBITS, 0, 0); // res = a*b BN_mul(res, a, b, ctx); printBN("a * b = ", res); // res = a^b mod n BN_mod_exp(res, a, b, n, ctx); printBN("a^b mod n = ", res); return 0; } Compiliation. We can use the following command to compile bn sample.c (the character after - is the letter l, not the number 1; it tells the compiler to use the crypto library). $ gcc bn_sample.c -lcrypto To avoid mistakes, please avoid manually typing the numbers used in the following tasks. Instead, copy and paste the numbers from this PDF file. Again, you should describe your steps and include your code and running results. 4.3 Task 12: Deriving the Private Key [3 Marks] Let p, q, and e be three prime numbers. Let n = p ∗ q. We will use (e, n) as the public key. Please calculate the private key d. The hexadecimal values of p, q, and e are listed in the following. It should be noted that although p and q used in this task are quite large numbers, they are not large enough to be secure. We intentionally make them small for the sake of simplicity. In practice, these numbers should be at least 1024 bits long (the one used here are only 128 bits). p = F7E75FDC469067FFDC4E847C51F452DF q = E85CED54AF57E53E092113E62F436F4F e = 0D88C3 4.4 Task 13: Encrypting a Message [4 Marks] Let (e, n) be the public key. Please encrypt the message “A top secret!” (the quotations are not included). We need to convert this ASCII string to a hex string, and then convert the hex string to a BIGNUM using the hex-to-bn API BN hex2bn(). The following python command can be used to convert a plain ASCII string to a hex string. $ python -c 'print("A top secret!".encode("hex"))' 4120746f702073656372657421 The public keys are listed in the followings (hexadecimal). We also provide the private key d to help you verify your encryption result. n = DCBFFE3E51F62E09CE7032E2677A78946A849DC4CDDE3A4D0CB81629242FB1A5 e = 010001 (this hex value equals to decimal 65537) M = A top secret! d = 74D806F9F3A62BAE331FFE3F0A68AFE35B3D2E4794148AACBC26AA381CD7D30D 4.5 Task 14: Decrypting a Message [4 Marks] The public/private keys used in this task are the same as the ones used in the previous task. Please decrypt the following ciphertext C, and convert it back to a plain ASCII string. C = 8C0F971DF2F3672B28811407E2DABBE1DA0FEBBBDFC7DCB67396567EA1E2493F You can use the following python command to convert a hex string back to to a plain ASCII string. $ python -c ’print("4120746f702073656372657421".decode("hex"))’ A top secret! 4.6 Task 15: Signing a Message [4 Marks] The public/private keys used in this task are the same as the ones used in the previous task. Please generate a signature for the following message (please directly sign this message, instead of signing its hash value): M = I owe you $2000. Please make a slight change to the message M, such as changing $2000 to $3000, and sign the modified message. Compare both signatures and describe what you observe. 4.7 Task 16: Verifying a Message [4 Marks] Bob receives a message M = “Launch a missile.” from Alice, with her signature S. We know that Alice’s public key is (e, n). Please verify whether the signature is indeed Alice’s or not. The public key and signature (hexadecimal) are listed in the following: M = Launch a missle. S = 643D6F34902D9C7EC90CB0B2BCA36C47FA37165C0005CAB026C0542CBDB6802F e = 010001 (this hex value equals to decimal 65537) n = AE1CD4DC432798D933779FBD46C6E1247F0CF1233595113AA51B450F18116115 Suppose that the signature in is corrupted, such that the last byte of the signature changes from 2F to 3F, i.e, there is only one bit of change. Please repeat this task, and describe what will happen to the verification process. 4.8 Task 17: Manually Verifying an X.509 Certificate [5 Marks] Figure 5: Illustration Figure of digital certificate verfication. In this task, we will manually verify an X.509 certificate using our program. An X.509 contains data about a public key and an issuer’s signature on the data (see Fig. 5 for illustration). We will download a real X.509 certificate from a web server, get its issuer’s public key, and then use this public key to verify the signature on the certificate. Step 1: Download a certificate from a real web server. We use the www.example.org server in this document. At this point, you should use the web server www.chase.com in your assignment. We can download certificates using browsers or use the following command: $ openssl s_client -connect www.example.org:443 -showcerts Certificate chain 0 s:/C=US/ST=California/L=Los Angeles/O=Internet Corporation for Assigned Names and Numbers/OU=Technology/CN=www.example.org i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA -----BEGIN CERTIFICATE----- MIIF8jCCBNqgAwIBAgIQDmTF+8I2reFLFyrrQceMsDANBgkqhkiG9w0BAQsFADBw MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3 ...... wDSiIIWIWJiJGbEeIO0TIFwEVWTOnbNl/faPXpk5IRXicapqiII= -----END CERTIFICATE----- 1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA -----BEGIN CERTIFICATE----- MIIEsTCCA5mgAwIBAgIQBOHnpNxc8vNtwCtCuF0VnzANBgkqhkiG9w0BAQsFADBs MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3 ...... cPUeybQ= -----END CERTIFICATE----- The result of the command contains two certificates. The subject field (the entry starting with s:) of the certificate is www.example.org, i.e., this is www.example.org’s certificate. The issuer field (the entry starting with i:) provides the issuer’s information. The subject field of the second certificate is the same as the issuer field of the first certificate. Basically, the second certificate belongs to an intermediate CA. In this task, we will use CA’s certificate to verify a server certificate. If you only get one certificate back using the above command, that means the certificate you get is signed by a root CA. Root CAs’ certificates can be obtained from the Firefox browser installed in our pre-built VM. Go to the Edit → Preferences → Privacy and then Security → View Certificates. Search for the name of the issuer and download its certificate. Copy and paste each of the certificate (the text between the line containing “Begin CERTIFICATE” and the line containing “END CERTIFICATE”, including these two lines) to a file. Let us call the first one c0.pem and the second one c1.pem. Step 2: Extract the public key (e, n) from the issuer’s certificate. Openssl provides commands to extract certain attributes from the x509 certificates. We can extract the value of n using -modulus. There is no specific command to extract e, but we can print out all the fields and can easily find the value of e. For modulus (n): $ openssl x509 -in c1.pem -noout -modulus Print out all the fields, find the exponent (e): $ openssl x509 -in c1.pem -text -noout Step 3: Extract the signature from the server’s certificate. There is no specific openssl command to extract the signature field. However, we can print out all the fields and then copy and paste the signature block into a file (note: if the signature algorithm used in the certificate is not based on RSA, you can find another certificate). $ openssl x509 -in c0.pem -text -noout ... Signature Algorithm: sha256WithRSAEncryption 84:a8:9a:11:a7:d8:bd:0b:26:7e:52:24:7b:b2:55:9d:ea:30: 89:51:08:87:6f:a9:ed:10:ea:5b:3e:0b:c7:2d:47:04:4e:dd: ...... 5c:04:55:64:ce:9d:b3:65:fd:f6:8f:5e:99:39:21:15:e2:71: aa:6a:88:82 We need to remove the spaces and colons from the data, so we can get a hex-string that we can feed into our program. The following command commands can achieve this goal. The tr command is a Linux utility tool for string operations. In this case, the -d option is used to delete “:” and “space” from the data. $ cat signature | tr -d '[:space:]:' 84a89a11a7d8bd0b267e52247bb2559dea30895108876fa9ed10ea5b3e0bc7 ...... 5c045564ce9db365fdf68f5e99392115e271aa6a8882 Step 4: Extract the body of the server’s certificate. A Certificate Authority (CA) generates the signature for a server certificate by first computing the hash of the certificate, and then sign the hash. To verify the signature, we also need to generate the hash from a certificate. Since the hash is generated before the signature is computed, we need to exclude the signature block of a certificate when computing the hash. Finding out what part of the certificate is used to generate the hash is quite challenging without a good understanding of the format of the certificate. X.509 certificates are encoded using the ASN.1 (Abstract Syntax Notation.One) standard, so if we can parse the ASN.1 structure, we can easily extract any field from a certificate. Openssl has a command called asn1parse, which can be used to parse a X.509 certificate. $ openssl asn1parse -i -in c0.pem 0:d=0 hl=4 l=1522 cons: SEQUENCE 4:d=1 hl=4 l=1242 cons: SEQUENCE # 8:d=2 hl=2 l= 3 cons: cont [ 0 ] 10:d=3 hl=2 l= 1 prim: INTEGER :02 13:d=2 hl=2 l= 16 prim: INTEGER :0E64C5FBC236ADE14B172AEB41C78CB0 ... ... 1236:d=4 hl=2 l= 12 cons: SEQUENCE 1238:d=5 hl=2 l= 3 prim: OBJECT :X509v3 Basic Constraints 1243:d=5 hl=2 l= 1 prim: BOOLEAN :255 1246:d=5 hl=2 l= 2 prim: OCTET STRING [HEX DUMP]:3000 1250:d=1 hl=2 l= 13 cons: SEQUENCE * 1252:d=2 hl=2 l= 9 prim: OBJECT :sha256WithRSAEncryption 1263:d=2 hl=2 l= 0 prim: NULL 1265:d=1 hl=4 l= 257 prim: BIT STRING The field starting from # is the body of the certificate that is used to generate the hash; the field starting from * is the signature block. Their offsets are the numbers at the beginning of the lines. In our case, the certificate body is from offset 4 to 1249, while the signature block is from 1250 to the end of the file. For X.509 certificates, the starting offset is always the same (i.e., 4), but the end depends on the content length of a certificate. We can use the -strparse option to get the field from the offset 4, which will give us the body of the certificate, excluding the signature block. $ openssl asn1parse -i -in c0.pem -strparse 4 -out c0_body.bin -noout Once we get the body of the certificate, we can calculate its hash using the following command: $ sha256sum c0_body.bin Step 5: Verify the signature. Now we have all the information, including the CA’s public key, the CA’s signature, and the body of the server’s certificate. We can run our own program to verify whether the signature is valid or not. Openssl does provide a command to verify the certificate for us, but you are required to use their own programs to do so, otherwise, they get zero credit for this task. 5 Pseudo Random Number Generation [21 Marks] 5.1 Task 18: Generate Encryption Key in a Wrong Way [4 Marks] To generate good pseudo random numbers, we need to start with something that is random; otherwise, the outcome will be quite predictable. The following program uses the current time as a seed for the pseudo random number generator. #include #include #include #define KEYSIZE 16 void main() { int i; char key[KEYSIZE]; printf("%lld ", (long long) time(NULL)); srand (time(NULL)); for (i = 0; i< KEYSIZE; i++){ key[i] = rand()%256; printf("%.2x", (unsigned char)key[i]); } printf(" "); } The library function time() returns the time as the number of seconds since the Epoch, 1970-01-01 00:00:00 +0000 (UTC). Run the code above, and describe your observations. Then, comment out Line 13 (start with “srand”), run the program again, and describe your observations. Use the observations in both cases to explain the purpose of the srand() and time() functions in the code. 5.2 Task 19: Guessing the Key [5 Marks] On April 17, 2018, Alice finished her tax return, and she saved the return (a PDF file) on her disk. To protect the file, she encrypted the PDF file using a key generated from the program described in Task 18. She wrote down the key in a notebook, which is securely stored in a safe. A few month later, Bob broke into her computer and gets a copy of the encrypted tax return. Since Alice is CEO of a big company, this file is very valuable. Bob cannot get the encryption key, but by looking around Alice’s computer, he saw the key-generation program, and suspected that Alice’s encryption key may be generated by the program. He also noticed the timestamp of the encrypted file, which is “2018-04-17 23:08:49”. He guessed that the key may be generated within a two-hour window before the file was created. Since the file is a PDF file, which has a header. The beginning part of the header is always the version number. Around the time when the file was created, PDF-1.5 was the most common version, i.e., the header starts with %PDF-1.5, which is 8 bytes of data. The next 8 bytes of the data are quite easy to predict as well. Therefore, Bob easily got the first 16 bytes of the plaintext. Based on the meta. data of the encrypted file, he knows that the file is encrypted using aes-128-cbc. Since AES is a 128-bit cipher, the 16-byte plaintext consists of one block of plaintext, so Bob knows a block of plaintext and its matching ciphertext. Moreover, Bob also knows the Initial Vector (IV) from the encrypted file (IV is never encrypted). Here is what Bob knows: Plaintext: 255044462d312e350a25d0d4c5d80a34 Ciphertext: d06bf9d0dab8e8ef880660d2af65aa82 IV: 09080706050403020100A2B2C2D2E2F2 Your job is to help Bob find out Alice’s encryption key, so you can decrypt the entire document. You should write a program to try all the possible keys. If the key was generated correctly, this task will not be possible. However, since Alice used time() to seed her random number generator, you should be able to find out her key easily. You can use the date command to print out the number of seconds between a specified time and the Epoch, 1970-01-01 00:00:00 +0000 (UTC). See the following example. $ date -d "2018-04-15 15:00:00" +%s 1523818800 Note that you can do it using Python, C or whatever language you prefer. 5.3 Task 20: Measure the Entropy of Kernel [4 Marks] In the virtual world, it is difficult to create randomness, i.e., software alone is hard to create random numbers. Most systems resort to the physical world to gain the randomness. Linux gains the randomness from the following physical resources: void add_keyboard_randomness(unsigned char scancode); void add_mouse_randomness(__u32 mouse_data); void add_interrupt_randomness(int irq); void add_blkdev_randomness(int major); The first two are quite straightforward to understand: the first one uses the timing between key presses; the second one uses mouse movement and interrupt timing; the third one gathers random numbers using the interrupt timing. Of course, not all interrupts are good sources of randomness. For example, the timer interrupt is not a good choice, because it is predictable. However, disk interrupts are a better measure. The last one measures the finishing time of block device requests. The randomness is measured using entropy, which is different from the meaning of entropy in the infor-mation theory. Here, it simply means how many bits of random numbers the system currently has. You can find out how much entropy the kernel has at the current moment using the following command. $ cat /proc/sys/kernel/random/entropy_avail Let us monitor the change of the entropy by running the above command via watch, which executes a program periodically, showing the output in fullscreen. The following command runs the cat program every 0.1 second. $ watch -n .1 cat /proc/sys/kernel/random/entropy_avail Please run the above command. While it is running, move your mouse, click your mouse, type some things, read a large file, visit a website. What activities increases the entropy significantly. Please describe your observation in your report. 5.4 Task 21: Get Pseudo Random Numbers from /dev/random [4 Marks] Linux stores the random data collected from the physical resources into a random pool, and then uses two devices to turn the randomness into pseudo random numbers. These two devices are /dev/random and/dev/urandom. They have different behaviors. The /dev/random device is a blocking device. Namely, every time a random number is given out by this device, the entropy of the randomness pool will be decreased. When the entropy reaches zero, /dev/random will block, until it gains enough randomness. Let us design an experiment to observe the behavior. of the /dev/random device. We will use the cat command to keep reading pseudo random numbers from /dev/random. We pipe the output to hexdump for nice printing. $ cat /dev/random | hexdump Please run the above command and at the same time use the watch command to monitor the entropy. What happens if you do not move your mouse or type anything? Then, randomly move your mouse and see whether you can observe any difference. Please describe and explain your observations. Question: If a server uses /dev/random to generate the random session key with a client. Please describe how you can launch a Denial-Of-Service (DOS) attack on such a server. 5.5 Task 22: Get Random Numbers from /dev/urandom [4 Marks] Linux provides another way to access the random pool via the /dev/urandom device, except that this device will not block. Both /dev/random and /dev/urandom use the random data from the pool to generate pseudo random numbers. When the entropy is not sufficient, /dev/random will pause, while /dev/urandom will keep generating new numbers. Think of the data in the pool as the “seed”, and as we know, we can use a seed to generate as many pseudo random numbers as we want. Let us see the behavior. of /dev/urandom. We again use cat to get pseudo random numbers from this device. Please run the following command, and the describe whether moving the mouse has any effect on the outcome. $ cat /dev/urandom | hexdump Let us measure the quality of the random number. We can use a tool called ent, which has already been installed in our VM. According to its manual, “ent applies various tests to sequences of bytes stored in files and reports the results of those tests. The program is useful for evaluating pseudo-random number generators for encryption and statistical sampling applications, compression algorithms, and other applications where the information density of a file is of interest”. Let us first generate 1 MB of pseudo random number from/dev/urandom and save them in a file. Then we run ent on the file. Please describe your outcome, and analyze whether the quality of the random numbers is good or not. $ head -c 1M /dev/urandom > output.bin $ ent output.bin Theoretically speaking, the /dev/random device is more secure, but in practice, there is not much differ-ence, because the “seed” used by /dev/urandom is random and non-predictable (/dev/urandom does re-seed whenever new random data become available). A big problem of the blocking behavior. of /dev/random is that blocking can lead to denial of service attacks. Therefore, it is recommended that we use /dev/urandom to get random numbers. To do that in our program, we just need to read directly from this device file. The following code snippet shows how. #define LEN 16 // 128 bits unsigned char *key = (unsigned char *) malloc(sizeof(unsigned char)*LEN); FILE* random = fopen("/dev/urandom", "r"); fread(key, sizeof(unsigned char)*LEN, 1, random); fclose(random); Please modify the above code snippet to generate a 256-bit encryption key. Please compile and run your code; print out the numbers and include the screenshot in the report.
COMP 3500 Lecture 29: Mock Exam 2 Exercise 1 (Menti): When several processes access the same data concurrently and the outcome of the execution depends on the particular order in which the access takes place, is called critical condition. Exercise 2 (Menti): A condition variable is basically a container of threads that are waiting for a certain condition. Exercise 3 (Menti): Deadlock will arise if a process holding at least one resource is waiting to acquire additional resources held by other processes. Exercise 4 (Menti): If a resource allocation graph contains no cycles, then there is no deadlock. Exercise 5 (Menti): We can prevent the circular wait condition by A) using a safe sequence B) using mutual exclusion C) defining a linear ordering of resource types D) all of the mentioned Exercise 6 (Menti): The terms “deadlock prevention” and “deadlock avoidance” can be used interchangeably. Exercise 7 (Menti): Like semaphores, each condition variable is maintaining a counter to keep track of conditions. Exercise 8 (Menti): In ____ mode, the kernel runs on behalf of the user. A) user B) kernel C) hybrid D) all the above Exercise 9 (Menti): System calls provide an essential interface between a process and hardware components. Exercise 10 (Menti): Which one of the following ways periodically test for deadlocks? A) Prevention B) Avoidance C) Detection D) Deletion Exercise 11: There are three processes accessing four resources labeled from R1 to R5. Please use a resource allocation graph to detect if there is a possible deadlock. In case of a deadlock, can you change the order of the lock operators in a process to prevent the deadlock? void P1() { while (true) { lock(R2); lock(R1); lock(R3); /*Critical region*/ unlock(R2); unlock(R1); unlock(R3); } } void P2() { while (true) { lock(R5); lock(R3); lock(R4); /*Critical region*/ unlock (R5); unlock (R3); unlock (R4); } } void P3() { while (true) { lock(R4); lock(R1); /*Critical region*/ unlock (R4); unlock (R1); } } Exercise 12.1: An operating system (OS) is managing one resource type containing 174 instances. There are 37 current processes sharing these instances coordinated by the OS. Each process must access 6 instances. Will a potential deadlock occur? Please justify your answer. Exercise 12.2: An operating system (OS) is managing one resource type containing 413 instances. There are 100 current processes sharing these instances coordinated by the OS. Each process must access 5 instances. Is this a deadlock-free scenario? Please justify your answer. Exercise 13: The Cigarette Smoker's Problem. There are three smokers and one cigarette dealer. A smoker must make a cigarette before he or she can smoke it. Making a cigarette requires (1) tobacco, (2) paper, and (3) matches. Each smoker has one of the three items. Specifically, one smoker has tobacco, another has paper, and a third one has matches. The cigarette dealer has an infinite supply of tobacco, paper, and matches. The dealer randomly selects two of the three items and places them on a table. Then, the smoker who has the third item makes a cigarette and smoke. After the first smoker finishes smoking, the dealer will place other two of the three items on the table so that another smoker can enjoy cigarettes. Please use semaphores to synchronize the three smokers and the dealer.
Project 2: Listening Journal · Due 26 Apr by 20:00 · Points 100 · Submitting a file upload · File types pdf · Available 4 Mar at 0:00 - 3 Jun at 23:59 Weight: 40% Due date: Week 7, Friday, 8:00pm (AEST, local time in Melbourne) Assessment format: PDF submitted via Canvas upload Grading Criteria: Project 2 Grading Criteria Rubric PROJECT DESCRIPTION Choose one of the sonic environments you investigated in Project 1 to explore further across projects 2, 3 and 4. Conduct a multimodal sonic investigation into that place, with a focus on your listening perspective. Part 1: Sonic Investigation You must visit your chosen location at least three separate times. Note the date, time of day and weather conditions of your visits. Over the course of your investigations, you must use at least three different documentation techniques from the following list: · Soundwalk (audio-recorded, narrated and/or annotated afterwards) · Stationary audio recording · Field notes · Sound mapping · Quantitative sound measurements · Photography/video · Sketching or illustration ...or another method with prior discussion and written confirmation from one of your lecturers. You may choose to use multiple documentation techniques on the same visit, OR one different documentation technique per visit. Part 2: Analysis Review your documentation results and analyse each type of documentation separately (e.g. analysis of sound maps; analysis of sound measurements; analysis of soundwalks). Analyse and describe the key soundscape features identified through each documentation method, making reference to Amphoux's Signal, Ambience and Background Sonic Composition orders and CRESSON's Sonic Effects. You may wish to analyse other soundscape features relevant to your project interests (e.g. rhythms, types of sounds, spectral information, sound levels). Part 3: Reflection Critically reflect on and respond to the following questions: · What is your relationship to the place you investigated? How has that relationship developed across your project work? · Why did you choose to investigate this location? Make reference to your responses to Project 1. What was/is compelling to you about this place, and why did you chose this one to continue your investigations? · What did you learn about this sonic environment and your experience of it in your investigations? Make reference to your fieldwork where relevant. · Based on your listenings, what do you want to share, express, communicate or represent about this sonic environment? DELIVERABLES Compile the work you have done across Parts 1, 2 and 3 into a PDF listening journal of at least 2,000 words and upload via Canvas. Sound or video files should be included as hyperlinks to publicly-shared uploads. If you would like to submit your listening journal in another format (eg. blog posts), you must discuss this with and receive written confirmation from your lecturers prior to submission. Listening Journal submission must include the following: LEARNING OUTCOMES Upon successful completion of this assessment, you will be able to: · Apply your listening skills for identifying and analysing the principal components of a soundscape, · Integrate a range of electroacoustic and acoustic measurement technologies, with subjective listener assessment for soundscape research, · Critically discuss and present design ideas and findings about a soundscape using aural, oral and visual methods
BUSA8000 Techniques in Business Analytics, Session 1, 2024 Assessment Task Report - Topics 1 - 4 Due date Week 7 - 5th April 2024 11.55pm Weight (%) 30% Task description This is an individual assessment - Students will be given a business case to analyse. Submission Method On iLearn Feedback mechanism(s) TurnitIn Feedback available (anticipated date) 2-3 weeks post due date Links to Unit Learning Outcomes On successful completion you will be able to (ULO1, ULO2, ULO3): · Articulate the importance and application of data in a variety of contexts. · Apply methods for handling data in R. · Implement statistical learning algorithms in R. ASSESSMENT DESCRIPTION Nelson Mandela believed education was the most powerful weapon to change the world. But not every student has equal opportunities to learn. Effective policies and plans need to be enacted in order to make education more equitable—and perhaps your innovative data analysis will help reveal the solution. Current research shows educational outcomes are far from equitable. The imbalance was exacerbated by the COVID-19 pandemic. There's an urgent need to better understand and measure the scope and impact of the pandemic on these inequities. Education technology company LearnPlatform. was founded in 2014 with a mission to expand equitable access to education technology for all students and teachers. LearnPlatform’s comprehensive edtech effectiveness system is used by districts and states to continuously improve the safety, equity, and effectiveness of their educational technology. LearnPlatform. does so by generating an evidence basis for what’s working and enacting it to benefit students, teachers, and budgets. In this assessment, you’ll work to uncover trends in digital learning. Accomplish this with data analysis about how engagement with digital learning relates to factors like district demographics, broadband access, and state/national level policies and events. Then, submit a report over Turnitin to propose your best solution to these educational inequities. Your submissions will inform. policies and practices that close the digital divide. With a better understanding of digital learning trends, you may help reverse the long-term learning loss among America’s most vulnerable, making education more equitable. Skills in focus for this assessment · Basic syntax knowledge of R for operations and functions. · Familiarity of R packages and relevance to data manipulation and visualisation. · Data importing from multiple sources. · Understanding how to convert data types. · Handling missing values, techniques for detecting, removing or imputing missing values. · Ability to identify and correct errors in data such as outliers or inconsistent entries. · Graphical representation skills for various types of visualisations. · Understanding the basic statistics for data analysis. · Critical thinking for analysing data, identify patterns or trends and draw meaningful conclusions. · Interpretation of results of data analysis and visualisations. · Familiarity of R Studio development environment for R programing. ASSESSMENT INSTRUCTIONS Step1: Use digital learning data to analyse the impact of COVID-19 on student learning. Step 2: You will prepare a report to explore: (1) the state of digital learning in 2020 (2) how the engagement of digital learning relates to factors such as district demographics, broadband access, and state/national level policies and events. Below are some examples of questions can guide your analysis: · What is the picture of digital connectivity and engagement in 2020? · What is the effect of the COVID-19 pandemic on online and distance learning, and how might this also evolve in the future? · How does student engagement with different types of education technology change over the course of the pandemic? · How does student engagement with online learning platforms relate to different geography? Demographic context (e.g., race/ethnicity, ESL, learning disability)? Learning context? Socioeconomic status? · Do certain state interventions, practices or policies (e.g., stimulus, reopening, eviction moratorium) correlate with the increase or decrease online engagement? Requirements: 1. In your report, describe the data cleaning and wrangling process, justify any transformation of data and choice of analysis/visualisation techniques, clearly explain your transformation, visualisations, findings and provide valid conclusions. 2. You are supposed to conduct your analysis mainly using what we have learned in the first four modules (i.e. usage of more advanced statistical/computational techniques is possible but not necessary, and won’t be evaluated for grading purpose). 3. You must prepare your report using the provided RMD template. Submit both your RMD file and the Word Doc or PDF generated from R markdown. Make sure your submitted RMD file can be run successfully on a different machine and all results presented can be reproducible. Refer to the RMD template and iLearn for further instructions on the report requirements. ASSESSMENT INSTRUCTIONS ASSESS TIPS & FAQs TIPS Start with a Plan: Before diving into coding, outline your analysis plan. Understand the objectives, the data you're working with, and the steps you might need to take to reach your conclusions. Understand Your Data: Spend time exploring and understanding your data. Use summary statistics and initial visualisations to get a feel for the data's structure, variables, and potential quirks. Keep Your Code Organized: Comment your code extensively and use consistent naming conventions for variables and functions. This makes your code easier to understand and debug, both for you and others. Break Down the Task: Divide your assignment into smaller, manageable tasks (e.g., data import, cleansing, transformation, analysis, visualization). Tackle each task one at a time. FAQs Q: How do I choose the right type of visualisation? A: The choice of visualisation depends on the nature of your data (categorical vs. numerical) and the story you want to tell. Q: How many visualisations should I include in my report? A: Include visualisations that add value to your analysis and help convey your findings clearly. Quality over quantity; each visualisation should have a clear purpose. Q: Can I use other packages beyond what was learned? A: Absolutely! R has a vast ecosystem of packages. If you find one that suits your needs better, feel free to use it. Just ensure you understand it well and explain your choice in your assignment. MENT DESCRIPTION USE OF RESOURCES AND TECHNOLOGIES INCLUDING GENERATIVE ARTIFICIAL INTELLIGENCE For this assessment, students are permitted to use generative artificial intelligence tools (GAITs e.g., ChatGPT) to: · clarify concepts, theories, ideas, etc., discussed in class · generate preliminary ideas for writing and coding · edit a working draft of the assessment · read and summarise research and supporting evidence for the assessment Students are not permitted to use GAITs to · generate definitions or writing used in their final submission. · produce counter-arguments or refine thinking on their final submission · generate complete R code in their final submission. Any of these actions will constitute and be treated as a breach of academic integrity.
