Breakdown the Fundamentals

In deep learning, generally, to approach the optimal value, gradient descent is applied to the weights, and optimization is achieved by running many many epochs with large datasets. The process is computationally intensive, and more than often you need to fit a dataset multiple times, thus, an efficient and fast optimization method helping you to get to the optimal more quickly is of great importance.

In this article, we will go through some general techniques applied in the optimization process.

Image for post
Image for post
Photo by Tim Carey on Unsplash

Weighted Average

Firstly, let’s learn the idea of weighted average, it smoothes values by given weights to past values.

Image for post
Image for post

Where t denotes the current timestamp, t-1 the last, and θ_t is the current value. …


Code it So You Understand it

In the last post, we have coded a deep dense neural network, but to have a better and more complete neural network, we would need it to be more robust and resistant to overfitting. The commonly applied method in a deep neural network, you might have heard, are regularization and dropout. In this article, we will together understand these 2 methods and implement them in python.

(we will directly use the function created in the last post in the following, if you get confused about some of the code, you might need to check the previous post)

Image for post
Image for post
Photo by fabio on Unsplash

Regularization

Regularization helps to prevent model from overfitting by adding an extra penalization term at the end of the loss function. …


Getting Started

Hands on Practice of Building a Deep Neural Network in Python

Image for post
Image for post
Photo by timJ on Unsplash

In last post, we’ve built a 1-hidden layer neural network with basic functions in python. To generalize and empower our network, in this post, we will build a n-layer neural network to do a binary classification task, in which n is customisable (it is recommended to go over my last introduction of neural network as the basics of theory would not be repeated here).

All images created on my own, referred images are source-added.

Weights Initialization

Firstly, weights need to be initialized for different layers. Note that in general, the input is not considered as a layer, but output is.

(For detailed training and testing process, please check…


Theory and Implementation Explained

One Hidden Layer NN

We will build a shallow dense neural network with one hidden layer, and the following structure is used for illustration purpose.

Before trying to understand this post, I strongly suggest you to go through my pervious implementation of logistic regression, as logistic regression can be seem as a 1-layer neural network and the basic concept is actually the same.

Image for post
Image for post

Where in the graph above, we have an input vector x = (x_1, x_2), containing 2 features and 4 hidden units a1, a2, a3 and a4, and output one value y_1 in [0, 1].(consider …


From Theory to Practice

Say we are doing a classic prediction task, where given a input vector with $n$ variables:

Image for post
Image for post

And to predict 1 response variable $y$ (may be the sales of next year, the house price, etc.), the simplest form is to use a linear regression to do the prediction with the formula:

Image for post
Image for post

Where $W$ is a column vector with $n$ dimension. Say now our question changed a bit, we hope to predict a probability, like what’s the probability of raining tomorrow? …


Understand Quantile Regression

For regression prediction tasks, not all time that we pursue only an absolute accurate prediction, and in fact, our prediction is always inaccurate, so instead of looking for an absolute precision, some times a prediction interval is required, in which cases we need quantile regression — that we predict an interval estimation of our target.

Loss Function

Fortunately, the powerful lightGBM has made quantile prediction possible and the major difference of quantile regression against general regression lies in the loss function, which is called pinball loss or quantile loss. There is a good explanation of pinball loss here, it has the formula:

Image for post
Image for post

Where y is the actual value, z is the prediction and 𝛕 is the targeted quantile. So the first sight of the loss function, we can see that besides when quantile equals to 0.5, the loss function is unsymmetrical. …


Flask + Lambda + API Gateway

The very first idea of creating my own app and deploying it on the cloud so that everyone could use it is super exciting to me, and this is what inspired me to write this post. If this idea also intrigues you, please follow through and from this post, you will learn how to deploy a python app step by step.

Firstly, you need an app

You will need to wrap your idea in an app, or say an API, which can process calls from the internet. An example is here. This is a flask app,

where the key lies in the app.py file, and this app receives your resume and help refer you internally. Notice that we don’t even need a docker file, the AWS Lambda is so light-weighted that you don’t even need to wrap your code in a container! …


A Trading Strategy Works on BTC and ETH

I have been trading on crypto currency for a few years and have watched loads of trading videos and tried to learn all kinds of trading patterns. But honestly speaking, none of these worked on me.

Reasons I Failed on Trading

No doubt I am not a pro-trader and got no strong business background. There are multiple reasons I found that make it hard for me to make ideal trades,

  • First, for most of us, I believe, we have to admit, are not professionals and most of our time can’t spend in front of the screen and watch every move of the market, thus we either enter the trade too early or too late. …


Deploy Your Container and Configure to Access AWS Resource

In this post, I will illustrate the process of deploying a docker container using AWS ECS and configure it to access other AWS resources. Now, let’s get right into the core.

Prerequisite: Basic understanding of docker, AWS and you need to have an AWS account with admin access.

Build a Container

If you already have a container ready to deploy, you can skip this part. If not, I wrote a simplest python project here, in which it has the most basic structure of a docker container:

ecs-example:
|_ model
| |_ run.py
| |_ sample.txt
|
|_ docker-entrypoint.sh
|_ Dockerfile
|_ requirements.txt

The main file run.py resides in the model folder and what it does is just to upload the sample.txt into s3 (S3 is AWS’s simple storage system to store…


Implementing a lined list in Python

Linked list is a list, but with each element pointing to the next element. So in a linked list, each node contains the current value of the node and a pointer to the next node.

Image for post
Image for post

Node

Like the graph shown above, the first node contains a value 2 and a pointer to the next node with value 4, and so on. So the implementation of a node would be:

Linked List

With the component of node implemented, let’s implement a linked list.

Add

So the first and most important function that a linked list need to fulfil is to append a new node to the end of the list. …

About

Jeremy Zhang

Hmm…I am a data scientist looking to catch up the tide…

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store