Non-Linear Support Vector Machines (SVM)
1. Introduction This blog post is about Support Vector Machines (SVM), but not only about SVMs. SVMs belong to the class of classification algorithms and are used to separate one …
Kernel Regression using Pyspark
1. Kernel Regression using Pyspark In a previous article I presented an implementation of a kernel denisty estimation using pyspark. It is thus not difficult to modify the algorithm to …
Nonparametric Density estimation using Spark
1. A Nonparametric Density implementation in Spark One of my previous blog post concerns about nonparametric density estimation. In this post i presented some Matlab code. An advantage of this …
Functional Regression with Spark
1. Functional Regression Let the covariate be an at least twice continuously differentiable random function defined wlog. on an interval and the corresponding the response. For simplicity we assume centered …
Functional Principal Component Analysis with Spark
1.) Functional Principal Component Analysis Let be a centered smooth random function in , with finite second moment . Without loss of generality we assume instead of some arbitrary compact …
3. A more sophisticated approach using Markov chains.
1. Generalize the Procedure The very simple Educated Guess Procedure is not only very simple, the procedure is also very unrealistic. At most the assumption that are independent and identically …
2. Coding the “Educated Guess Procedure”
1. Perform the Analyze To start with, we load the “rockyou.txt.tar.gz” password list using wget. I’m not sure if it is legal to provide a link for the list, therefore …
5. Running some tests
1. Test the Enviroment 1.1 Simulation of a Brownian Motion The purpose of the first notebook entry is to check if matplotlib is correctly installed. We simulate 20 Brownian Motions …
4. Install IPython Notebook for Remote Access and Hive
1. Requirements 2. Install Software In this section we will install some stuff which will make life easier. In constrast to Spark or Hadoop it is only required to install …
3. Build the Cluster
1. Requirements We need an SD Card with Lubuntu, Hadoop and Spark installed. 2. Build the Cluster 2.1 Clone the SD Card sudo shutdown 0 of your orangepi and remove …
2. Install Hadoop and Spark
1. Requirements An Orangepi with Lubuntu running, see this post for further instructions. 2.Install the Components 2.1 Update Java In fact Hadoop is not necessary for Spark. However, we will …
1. Setting up the OrangePi
1. Requirements OrangePI PC SD Card (larger than 8GB) 4.0*1.7 Powersuppy or USB Cable and Charger (min. 2A) WLAN Adapter or Ethernet Cable HDMI Cable and Monitor (only for setup) …
1. Thougts about Passwords
1. Introduction This Project is about making an educated guess to derive an unknown password based on hacked password lists (Google: “RockYou”). In this section we will introduce the mathematical …
Fourier Transform
1. The Fourier Transform The Fourier transform of an integrable function with is given by (1) Under suitable conditions, the inverse transform from to with is then given by …
Kernel based Estimators for Multivariate Densities and Functions
1. Kernel Functions In general, a kernel is an integrable function satisfying is symmetric (e.g. if ) . Popular univariate () kernel functions: Uniform: Epanechnikov: Gaussian: An easy way …
Estimating Multivariate Functions and Derivatives using Local Polynomial Regression in Matlab
1. Theoretical Background I want to start with some theory which will in the end lead us to the Matlab coding. If you are not interested how to derive the …