COM4509-6509_Assignment_1_UCard_XXXXXXXXX(1)Assignment 1 BriefDeadline: Tuesday, October 29, 2019 at 14:00 hrsNumber of marks available: 20Scope: Sessions 1 to 5How and what to submitA. Submit a Jupyter Notebook named COM4509-6509_Assignment_1_UCard_XXXXXXXXX.ipynb whereXXXXXXXXX refers to your UCard number.B. Upload the notebook file to MOLE before the deadline above.C. NO DATA UPLOAD: Please do not upload the data files used. We have a copy already.Assessment CriteriaBeing able to express an objective function and its gradients in matrix form.Being able to use numpy and pandas to preprocess a dataset.Being able to use numpy to build a machine learning pipeline for supervised learning.Late submissionsWe follow Departments guidelines about late submissions, i.e., a deduction of 5% of the mark each workingday the work is late after the deadline. NO late submission will be marked one week after the deadline becausewe will release a solution by then. Please read this link(https://sites.google.com/sheffield.ac.uk/compgtstudenthandbook/menu/assessment/late-submission?pli=1&authuser=1).Use of unfair meansAny form of unfair means is treated as a serious academic offence and action may be taken under theDiscipline Regulations. (from the MSc Handbook). Please carefully read this link(https://sites.google.com/sheffield.ac.uk/compgtstudenthandbook/menu/referencing-unfair-means?pli=1&authuser=1) on what constitutes Unfair Means if not sure.Regularisation for Linear RegressionRegularisation is a technique commonly used in Machine Learning to prevent overfitting. It consists on addingterms to the objective function such that the optimisation procedure avoids solutions that just learn the trainingdata. Popular techniques for regularisation in Supervised Learning include Lasso Regression, RidgeRegression and the Elastic Net.In this Assignment, you will be looking at Ridge Regression and devising equations to optimise the objectivefunction in Ridge Regression using two methods: a closed-form derivation and the update rules for stochasticgradient descent. You will then use those update rules for making predictions on a Air Quaility dataset.2019/10/27 COM4509-6509_Assignment_1_UCard_XXXXXXXXX(1)localhost:8888/notebooks/COM4509-6509_Assignment_1_UCard_XXXXXXXXX(1).ipynb 2/7Ridge RegressionLet us start with a data set for training , where the vector and is the designmatrix from Lab 3, this is,Our predictive model is going to be a linear modelwhere .The objetive function we are going to use has the following formwhere is known as the regularisation parameter.The first term on the right-hand side (rhs) of the expression for is very similar to the least-squaresobjective function we have seen before, for example in Lab 3. The only difference is on the term that we useto normalise the objective with respect to the number of observations in the dataset.The first term on the rhs is what we call the fitting term whereas the second term in the expression is theregularisation term. Given , the two terms in the expression have different purposes. The first term is lookingfor a value of that leads the squared-errors to zero. While doing this, can take any value and lead to asolution that it is only good for the training data but perhaps not for the test data. The second term isregularising the behavior of the first term by driving the towards zero. By doing this, it restricts the possibleset of values that might take according to the first term. The value that we use for will allow a compromisebetween a value of that exactly fits the data (first term) or a value of that does not grow too much (secondterm).This type of regularisation has different names: ridge regression, Tikhonov regularisation or normregularisation.Question 1: in matrix form (2 marks)Write the expression for in matrix form. Include ALL the steps necessary to reach the expression.Question 1 AnswerWrite your answer to the question in this box.Optimising the objective function with respect toThere are two ways we can optimise the objective function with respect to . The first one leads to a closedform expression for and the second one using an iterative optimisation procedure that updates the value ofat each iteration by using the gradient of the objective function with respect to ,2019/10/27 COM4509-6509_Assignment_1_UCard_XXXXXXXXX(1)localhost:8888/notebooks/COM4509-6509_Assignment_1_UCard_XXXXXXXXX(1).ipynb 3/7at eac te at o by us g t e g ad e t o t e object e u ct o t espect to ,where is the learning rate parameter and is the gradient of the objective function.Question 2: Derivative of wrt (2 marks)Find the closed-form expression for by taking the derivative of with respect to , equating to zeroand solving for . Write the expression in matrix form.Also, write down the specific update rule for by using the equation above.Question 2 AnswerWrite your answer to the question in this box.Using ridge regression to predict air qualityOur dataset comes from a popular machine learning repository that hosts open source datasets for educationaland research purposes, the UCI Machine Learning Repository (https://archive.ics.uci.edu/ml/index.php). We aregoing to use ridge regression for predicting air quality. The description of the dataset can be found here(https://archive.ics.uci.edu/ml/datasets/Air+Quality).In [ ]:In [ ]:We can see some of the rows in the datasetIn [ ]:The target variable corresponds to the CO(GT) variable of the first column. The following columns correspondto the variables in the feature vectors, e.g., PT08.S1(CO) is up until AH which is . The original datasetalso has a date and a time columns that we are not going to use in this assignment.Before designing our predictive model, we need to think about three stages: the preprocessing stage, thetraining stage and the validation stage. The three stages are interconnected and it is important to rememberthat the testing data that we use for validation has to be set aside before preprocessing. Any preprocessing thatyou do has to be done only on the training data and several key statistics need to be saved for the test stage.𝑥1 𝑥𝐷import podspods.util.download_url(https://archive.ics.uci.edu/ml/machine-learning-databases/00360/AirQualityUCimport zipfilezip = zipfile.ZipFile(./AirQualityUCI.zip, r)for name in zip.namelist(): zip.extract(name, .)# The .csv version of the file has some typing issues, so we use the excel versionimport pandas as pdair_quality = pd.read_excel(./AirQualityUCI.xlsx, usecols=range(2,15))air_quality.sample(5)2019/10/27 COM4509-6509_Assignment_1_UCard_XXXXXXXXX(1)localhost:8888/notebooks/COM4509-6509_Assignment_1_UCard_XXXXXXXXX(1).ipynb 4/7Separating the dataset into training and test before any preprocessing has happened help us to recreate thereal world scenario where代写COM4509-6509、代做Python、代写pipe we will deploy our system and for which the data will come without anypreprocessing.We are going to use hold-out validation for testing our predictive model so we need to separate the dataset intoa training set and a test set.Question 3: Splitting the dataset (1 mark)Split the dataset into a training set and a test set. The training set should have 70% of the total observationsand the test set, the 30%. For making the random selection make sure that you use a random seed thatcorresponds to the last five digits of your student UCard. Make sure that you comment your code.Question 3 AnswerIn [ ]:Preprocessing the dataThe dataset has missing values tagged with a -200 value. Before doing any work with the training data, we wantto make sure that we deal properly with the missing values.Question 4: Missing values (3 marks)Make some exploratory analysis on the number of missing values per column in the training data.Remove the rows for which the target feature has missing values. We are doing supervised learning so weneed all our data observations to have known target values.Remove features with more than 20% of missing values. For all the other features with missing values, usethe mean value of the non-missing values for imputation.Question 4 AnswerIn [ ]:Question 5: Normalising the training data (2 marks)Now that you have removed the missing data, we need to normalise the input vectors.Explain in a sentence why do you need to normalise the input features for this dataset.Normalise the training data by substracting the mean value for each feature and dividing the result by thestandard deviation of each feature. Keep the mean values and standard deviations, you will need them attest time.Question 5 AnswerWrite your explanation in this box# Write your code here# Write your code here2019/10/27 COM4509-6509_Assignment_1_UCard_XXXXXXXXX(1)localhost:8888/notebooks/COM4509-6509_Assignment_1_UCard_XXXXXXXXX(1).ipynb 5/7In [ ]:Training and validation stagesWe have now curated our training data by removing data observations and features with a large amount ofmissing values. We have also normalised the feature vectors. We are now in a good position to work ondeveloping the prediction model and validating it. We will use both the closed form expression for andgradient descent for iterative optimisation.We first organise the dataframe into the vector of targets and the design matrix .In [ ]:Question 6: training with closed form expression for (3 marks)To find the optimal value of using the closed form expression that you derived before, we need to know thevalue of the regularisation parameter in advance. We will determine the value by using part of the trainingdata for finding the parameters and another part of the training data to choose the best from a set ofpredefined values.Use np.logspace(start, stop, num) to create a set of values for in log scale. Use the followingparameters start=-3 , stop=2 and num=20 .Randomly split the training data into what is properly called the training set and the validation set. Asbefore, make sure that you use a random seed that corresponds to the last five digits of your studentUCard. Use 70% of the data for the training set and 30% of the data for the validation set.For each value that you have for from the previous step, use the training set to compute and thenmeasure the mean-squared error (MSE) over the validation data. After this, you will have num=20 MSEvalues. Choose the value of that leads to the lower MSE and save it. You will use it at the test stage.What was the best value of ? Is there any explanation for that?Question 6 AnswerIn [ ]:Write your answer to the last question here.Question 7: validation with the closed form expression for (2 marks)We are going to deal now with the test data to perform the validation of the model. Remember that the test datamight also contain missing values in the target variable and in the input features.Remove the rows of the test data for which the labels have missing values.𝐰# Write your code here# Write your code here to get y and Xy =X =# Write your code here2019/10/27 COM4509-6509_Assignment_1_UCard_XXXXXXXXX(1)localhost:8888/notebooks/COM4509-6509_Assignment_1_UCard_XXXXXXXXX(1).ipynb 6/7If you remove any feature at the training stage, you also need to remove the same features from the teststage.Replace the missing values on each feature variables with the mean value you computed in the trainingdata.Normalise the test data using the means and standard deviations computed from the training dataCompute again for the value of that best performed on the validation set using ALL the training data(not all the training set).Report the MSE on the preprocessed test data and an histogram with the absolute error.Does the regularisation have any effect on the model? Explain your answer.Question 7 Answer𝐰 𝛼In [ ]:Write the explanation to your answer here.Question 8: training with gradient descent and validation (5marks)Use gradient descent to iteratively compute the value of . Instead of using all the training set to computethe gradient, use a subset of datapoints in the training set. This is sometimes called minibatch gradientdescent where is the size of the minibacth. When using gradient descent with minibatches, you need to findthe best values for three parameters: , the learning rate, , the number of datapoints in the minibatch and ,the regularisation parameter.As you did on Question 6, create a grid of values for the parameters and using np.logspace and agrid of values for using np.linspace. Because you need to find three parameters, start with num=5 andsee if you can increase it.Use the same training set and validation set that you used in Question 6.For each value that you have of , and from the previous step, use the training set to compute usingminibatch gradient descent and then measure the MSE over the validation data. For the minibatch gradientdescent choose to stop the iterative procedure after iterations.Choose the values of , and that lead to the lower MSE and save them. You will use them at the teststage.3 marks of out of the 5 marksUse the test set from Question 7 and provide the MSE obtained by having used minibatch training with thebest values for , and over the WHOLE training data (not only the training set).Compare the performance of the closed form solution and the minibatch solution. Are the performancessimilar? Are the parameters and similar in both approaches? Please comment on both questions.2 marks of out of the 5 marksQuestion 8 AnswerIn [ ]:# Write your code here# Write the code for your answer here2019/10/27 COM4509-6509_Assignment_1_UCard_XXXXXXXXX(1)localhost:8888/notebooks/COM4509-6509_Assignment_1_UCard_XXXXXXXXX(1).ipynb 7/7Write the answer to your last question here.转自：http://www.3daixie.com/contents/11/3444.html