What are Microservices

• What are Microservices

1. Presently a lot of hype!

1. Best described as:
2. An architectural style
3. An alternative to more traditional 'monolithic' applications
4. Decomposition of single system into a suite of small services, each running as independent proceses and intercommunicating via open protocols

       - With all the benefits / risks this implies.

• Definitions from the Experts

1. Developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.                 

      - Martin Fowler    

    

     2. Fine-grained SOA

      - Adrian Cockcroft - Netflix

• Microservices - Working Definition:

1. Composing a single application using a suite of small services

       - (rather than a single, monolithic application)

    2. ... each running as independent processes

       - (not merely modules / components within a single executable)

    3. ... intercommunicationg via open protocols

       - ( Like HTTP/REST, or messaging)

    4.  Separately written, deployed, scaled and maintained

       - ( potentially in different languages)

    5.  Services encapsulate business capability

      - ( rather than language constructs (classes, packages) as primary way to encapsulate.

    6. Services encapsulate business capability

      - ( rather than language constructs (classes, packages) as primary way to encapsulate.

    7. Services are independently replaceable and upgradable

• Microservices are not:

    1. The same as SOA

       - SOA is about integrating various enterprise applications.

         Microservices are mainly about decomposing single applications

    2. A silber bullet

       - The microservices approach involves drawbacks and risks

    3. New! You may be using microservices now and not konw it!

• Current Trends

    1. Twitter moved from Ruby/Rails monolith to Micorservices.

    2. Facebook moved from PHP monolith to Microservices.

    3. Netflix moved from Java monolith to Microservices.

• Microservices Example

    1. Consider a monolithic shopping cart application:

      - Web / mobile interfaces

      - Functions for:

          i) searching for products

          ii) Product catalog

          iii) Inventory management

          iv) Shopping cart

          v) Checkout

          vi) Fufillment

       - How would this look with microservices?

• Monlolithic Application Example

    1. Monolithic shopping cart application:

    2. Understanding the Monolithic Architecture

 

   3. New types of client applications

   4. New type of persistence / services

• Monolithic Challenges

    1. Single Codebase, Deployment, Versioning, Team Size
 

           

     2. Using Teams / Language Constructs

• Understanding the monolithic implementation

       1. Single application executable
          - Easy to comprehend, but not to digest.
          - Must be written in a single language.
       2. Modularity based on Program Language
          - Using the constructs available in that language (packages, classes, functions,                namespaces, frameworks)
         - Various storage / service technologies used
           => REBMS, Messaging, eMail, etc.

• Monolithic Advantages

1. Easy to comprehend(but not digest)
2. Easy to test as a single unit (up to a size limit)
3. Easy to deploy as a single unit.
4. Easy to manage (up to a size limit)
5. Easy to manage changes (up to a point)
6. Easy to scale (when care is taken)
7. Complexity managed by language constructs.

• Monolithic Drawbacks

   1. Language / Framework Lock
      - Entire app written with single technology stack. Cannot experiment / take                    advantage of emerging technologies
   2. Digestion
      - Single developer cannot digest a large codebase
      - Single team cannot manage a single large application   
        => Amazon's "2 Pizza" rule
   3. Deployment as single unit
     - Cannot independently deploy single change to single component.
     - Changes are "held-hostage" by other changes     

• Enter Microservices architecture

• Compoentization via Services

1. NOT language constructs.
2. Where services are smaill, independently deployabl applications
3. Forces the design of clear interfaces
4. Changes scoped to their affected service

• Microserices: Composed using suite of small services

1. Services are small, independently deployable applications

      - Not a single codebase
      - Not (necessarily) a single language / framework
      - Modularization not based on language / framework constructs

• Microservices: Communication based on lightweight protocols

1. HTTP, TCP, UDP, Messaging, etc.

      - Payloads: JSON, BSON, XML, Protocol Buffers, etc.
    2. Forces the design of clear interfaces
    3. Netflix's Cloud Native Architecture - Communicate via APIs
      - NOT Common Database

• Microservices: Services encapsulate business capabilities

1. Not based on technology stack
2. Vertical slices by business function (i.e. cart, catalog, checkout)
3. ...Though technology chunk also practical (email service)
4. Suitable for cross-functional teams

• Micoroservices: Services easily managed

1. Easy to comprehend, alter, test, version, deploy, manage, overhaul, replace

       - By small, cross-functional teams (or even individuals)

• Decentralized Governance

1. Use the right tool ( language, framework) for the job.
2. Services evolve at different speeds, deployed and managed according to different needs.
3. Make services be "Tolerant Readers"
4. Consumer-0Driven Contracts
5. Antithesis of ESB

      - Services are not Orchestrated, but Choreographed

• Polyglot Persistence

1. Freedom to use the best technology for the job

      - Don't assume single RDBMS is always best
      - Very controversial! Many DBAs will not like this!
         i) No pan-enterprise data model!
         ii) No transactions!

• Microservice Advantages

1. Easy to digest each service (difficult to comprehend whole)
2. VERY easy to test, deploy, manage, version, and scale single services
3. Change cycle decoupled
4. Easier to scale staff
5. No Language / Framework lock

• Challenges with Microservices

1. Complexity has moved out of the application, but into the operations layer

       - Fallacies of Distributed Computing

    2. Services may be unavailable
       - Never needed to worry about this in a monolith!
       - Design for failure, circuit breakers
        i) "Everything fails all the time" - Werner Vogels, CTO Amazon
       - Much more monitoring needed
 
     3. Remote calls more expensive than in-process calls

     4. Transactions: Must rely on eventual consistency over ACID

     5. Features span multiple services

     6. Change management becomes a different challenge
        - Need to consider the interaction of services
        - Dependency management / versions
 
      7. Refactoring Module Boundaries

• Fallacies of Distributed Computing

1. The network is reliable.
2. Latency is zero.
3. Bandwidth is infinite.
4. The network is secure.
5. Topology doesn't change.
6. Therre is noen administrator.
7. Transport cost is zero.
8. The network is homogeneous.

• How Do You Break a Monolith into Microservices?

1. Primary consideration: business functionality:

       - Noun-based (catalog, cart, customer)
       - Verb-based (search, checkout, shipping)
       - Single Responsibility Principle
         => http://programmer.97things.oreilly.com/wiki/index.php/The_Single_Responsibility_Principle
       - Bounded Context
         => http://martinfowler.com/kliki/BoundedContext.html

• How Micro is Micro?

1. Size  is not the compelling factor

      - Smaill enough for an individual developer to digest
      - Small enough to be built and managed by small team
        i) Amazon's two pizza raule
      - Documentation small enough to read and understand
        i) Social Security Act of 1935 - 63 pages
        ii) Affordable Care Act of 2010 - 906 pages

       - Dozens of secrets, not hundreds.
       - Predictable. Easy to experiment with

• Differences with SOA

1. SOA addresses integration between systems.

       - Microservices address individual applications
    2. SOA relies on orchestration.
       - Microservices rely on choreography

    3. SOA relies on smart integration technology, dumb services
       - Microservices rely on smart services, dum integration technology
       - Consider: Linux commands, pipes and filters

 

• Are Monoliths Always Bad?

1. Consider etsy.com

       - As of Rebruary 2013: 1.49 billion page views, 4,215,169 items sold, $94.7 million             of goods sold, 22+ million members
       - 150 developers deploy single WAR 60 tiems a day
       - Practices: Cl; push button deployment; good monitoring; developers deploy to             the site on the first day; VMs per developer; GitHub; Chef; IRC to controll                     releases; dashboards; no source control branches.

docker run -d --name kuard --publish 8080:8080 gcr.io/kuar-demo/kuard-amd64:1

curl http://localhost:800

docker stop kuard
docker rm kuard


docker run -d --name kuard --publish 8080:8080 --memory 200m --memory-swap 1G gcr.io/kuar-demo/kuard-amd64:1


docker run -d --name kuard --publish 8080:8080 --memory 200m --memory-swap 1G --cpu-shares 1024 gcr.io/kuar-demo/kuard-amd64:1


docker rmi
docker rmi

virtualbox uninstall

System Preferences -> security & privacy  -> Oracle America, inc allow

virtualbox install

minikube start

minikube start --kubernetes-version v1.17.0

minikube status

minikube stop
minikube delete

kubectl veersion

kubectl get componentstatuses

kubectl get nodes


kubectl describe nodes node-1

Microservice deployment to Kubernetes

https://hub.docker.com/r/karthequian/wishlist/

https://hub.docker.com/r/karthequian/wishlist-auth

https://hub.docker.com/r/karthequian/wishlist-catalog

Microservice deployment to Kubernetes

https://github.com/karthequian/wishlist

# Deploying our application to Kubernetes

We're ready to deploy our application to Kubernetes, but let's take a look at our assets.

## Goals:
1. View our sample application and containers
2. Take a look at our deployment file
3. Take a look at our alternate deployment file
4. Deploy our application into kubernetes and verify we can see our API's working.

## Goal 1
View the sample application here:

## Goal 2
To view the deployment file, take a look at wishlist-deployment.yaml

## Goal 3
To see another way to run the microservices, take a look at wishlist-deployment-alernate.yaml

## Goal 4
To run the microservice described in goal #1, from the current directory, run:

`kubectl create -f wishlist-deployment.yaml`

To verify that the deployment is online:
`kubectl get deployments`

To verify that the replica sets are running:
`kubectl get rs`

To verify that the pods are running:
`kubectl get pods`

To see the services:
`kubectl get services`

To interact with your API's in the minikube environment:
`minikube service wishlist-service`

# Wishlist deployment yaml

kind: Deployment

apiVersion: apps/v1

metadata:

  name: wishlist-deployment

  labels:

    app: wishlist

spec:

  replicas: 3 #We always want more than 1 replica for HA

  selector:

    matchLabels:

      app: wishlist

  template:

    metadata:

      labels:

        app: wishlist

    spec:

      containers:

      - name: wishlist #1st container

        image: karthequian/wishlist:1.0 #Dockerhub image

        ports:

        - containerPort: 8080 #Exposes the port 8080 of the container

        env:

        - name: PORT #Env variable key passed to container that is read by app

          value: "8080" # Value of the env port.

      - name: catalog #2nd container

        image: karthequian/wishlist-catalog:1.0

        ports:

        - containerPort: 8081

        env:

        - name: PORT

          value: "8081"

      - name: auth #3rd container

        image: karthequian/wishlist-auth:1.0

        ports:

        - containerPort: 8082

        env:

        - name: PORT

          value: "8082"

---

kind: Service

apiVersion: v1

metadata:

  name: wishlist-service

  namespace: default

spec:

  type: NodePort

  selector:

    app: wishlist

  ports:

  - name: wishlist-port

    protocol: TCP

    port: 8080

  - name: wishlist-auth-port

    protocol: TCP

    port: 8081

  - name: wishlist-catalog-port

    protocol: TCP

    port: 8082

Large Enterprise Application Experience

1. My Early Enterprise Experience
- Team built one big java WAR file
- Ops team deployed to the dev WebLogic Server
- Local development on WebLogic Servers run on own machines
- Environment set up to use the development database via data sources.


2. Ecommerce Catalog
- View the list of current products
- Understand their specifications
- Save into a with lisht that enables team collaboration
- Get a quote for the with list or go through the purchase process.

Three Modules
1) User Module
Responsible for user management, login, and profile management

2) Catalog Module
List of products in the company catalog

3) Wish List Module
API that the customers will user to create and view their wish lists

Our Monolith
- A single WAR file, and has one or more JAR files that provide all of the core functioinality
- Will likely have a common web tier to handle web requests and responses

Breaking This Up
Break the single application into three microservices.
- Auth
- Wish list functionality
- The catalog portion


Benefits of this Structure
- Each microservice has its own Rest APIs that can be userd by the other services

- Reorganize codebase in a more logical way

- Development team can split into smaller feature teams to work on th individual microservices.

- Code and API are more visible to business team

- Individual teams can develop, build, and test their applications locally-team is more productive up front

- Enables containuous integration pipeline

Containers allow you to user Kubernetes.

Transitioning to Microservices
- Addressed some of the microservice building block concrns
- Applied some of the basic principles from the twelve-factor pattern

Our microservices are running in Docker containers, so  let's Kube this up.

Microservice Patterns in Kubernnetes

1. Goal
1.1. Build reference architecture for microservices for Kubernetes

2. Architecture Groupings

2.1. Basic building block architectures
2.1.1. Deployment Patterns
2.1.2. Runtime patterns


3. Building Blocks

3.1. Covers the initial concerns
3.1.1. Codebase
3.1.1.1. Code stored in source control
- Example: Git, Perforce
3.1.1.2. Container images stored in an image repository
- Example: Docker Hub, Artifactory

3.1.1.3. Codebase Workflow
3.1.1.3.1. Push code to your source control.
3.1.1.3.2. Automated build is kicked off to build and run tests against code.
3.1.1.3.3. Bild container image; push to image repository.

3.1.1.4. Image Repository
3.1.1.4.1. Stores your code in an image
3.1.1.4.2. Need to decide early on
3.1.1.4.3. Follow your company guidelines

3.1.2. Dependencies

3.1.2.1. Applications modeled in Kubernetes as Deployments and pods
3.1.2.2. Single pod can have many containers inside
3.1.2.3. Commonly seen in Kubernetes: sidecar pattern

3.1.3. Dev/staging/production parity

3.1.3.1. Dev versus Prod in Kubernetes
3.1.3.1.1. Common Kubernetes deployment patterns
- Small footprint: different namespaces with different credentials for dev, staging, and production

- Large footprint: unique Kubernetes installation for dev, staging, and production


3.1.4. Admin processes.

3.1.4.1. Admin process containers tagged in a similar way to the running application
3.1.4.2. Containers run as Kubernetes jobs/chron job
3.1.4.3. Also run as a separate deployment


4. Deployment Patterns


4.1 Patterns around aplication deployment
4.1.1. Application configurations
4.1.1.1. Applications always have associated configuration to make the application work as expected

4.1.1.2. Application Configuration in Kubernetes
4.1.1.2.1. Two ways to store configs
- configMaps: for generic information(example: metadata, version)
- Secrets: for sensitive data(example:passwords)

4.1.1.2.2. Loaded into the pod via
- Environment variables
- Files


4.1.1.2.3. A Word on Secrets
- Secrets are a good start, but depending on the use case, might need something more like ashiCoprp Vault

4.1.2. Build, release, run
- Tag containers at build time with explicit version
- Don't use latest ta for production containers

4.1.2.1. Running in Kubernetes
- High-level constructs to run containers
- Deployments, DaemonSets, ReplicaSets
- Package management provided by Helm
- Adds revision control


4.1.3. Processes
4.1.3.1. Processes and Port Bindings
4.1.3.1.1. Processes
- Keep application stateless
- Goal: Alllow request to go to any container or server by default

4.1.3.1.2. Statelessness in Kubernetes
- Translated to Deployments and pods
- Deployments comprised of ReplicaSets, which are a collection of one or more pods

4.1.3.1.3. Word on StatefulSets
- Typically used to create persistent storage systems like a MySQK shard

4.1.3.2. Port Bindings
- Containers are implemented in pods
- Communicate to each other via well-defined ports



5. Runtime Patterns
Associating Resources in Kubernetes
- Everything is treated as a service, and configurations are stored in a ConfigMap or Secret

Scenario: Replace a Link to a Database

1) Create new database; ensure it's online and ready to accept connections.
2) Update configuration stored in the ConfigMaps or Secrets.
3) Kill the pods that were communicating with the old database.

When Kubernetes starts up the new pods, the new pods automatically pick up the new configuration, and you'll be using the new service.

If a pod is taking too much load or is receiving a lot of requests, it's easy to scale the number of pods in Kubernetes by adding more to the replica set.

Scaling Kubernetes to handle more traffic is one of the strengths of the platform.



5.1. Patterns for live running systems
5.1.1. Backing services
5.1.2. Features around concurreny
5.1.3. Disposability
The abliity to maximize robustness with fast startup and graceful shutdown

Containers
- Start up fast and run efficiently
- Have all required tooling built inside of the container image

Containers to Pods to Replicas
- Kubernetes manages your containers in pods, which are managed with ReplicaSets
- If one of the pods goes offline, the ReplicaSet will automatically start up a new pod to take its place

Advantages to Users
- From a user perspective, the application will still function, and the user won't see any downtime

5.1.4. Log management

5.1.4.1. Logging
- Treat logs as streams: execution environment handles the logs
- Common to user a log router (Beats/Fluentd) to save the logs to a service(Elasticsearch/ Splunk)
- Kubernetes makes this process easy


6. Some Assembly Required

6.1 Basic Knowledge of Kubernetes required
6.2. Watch the Learning Kubernetes course in the library if needed

Recurrent Neural Networks

-> google translator relies heavily on recurrent neural networks
-> we can use recurrent neural networks to make time series analysis(주식 분석)

Turing-Test: a computer passes the Turing-test if a human is unable to distinguish the computer from a human in a blind test

~ recurrent neural networks are able to pass this test: a well-trained recurrent network is able to,, understand" English for example

LEARN LANGUAGE MODELS!!

We would like to make sure that the network is able to learn connections in the data even when they are far away from each other

,, I am from Hungary. Lorem ipsum dolor sit amet, consetetur adipiscing elit, sed do eiusmod tempor incididnt

Recurrent neural networks are able to deal with relationships for away from eah other

~it is able to guess the last word: humgarian


Combining convolutional neural networks with recurrent neural networks is quite powerful

~ we can generate image descriptions with this hibrid approach


With Multilayer Neural Networks ( or deep networks) we make predictions independent of each other
p(t) is not correlated with p(t-1) or p(t-2)...

-> training examples are independent of each other
               Tigers, elephants, cats ..  nothing to do with each other

THESE PREDICTIONS ARE INDEPENDENT !!!!

With Recurrent Neural Networks we can predict the next word in a given sentence:
     it is important in natural language processing ~or we want to predict the stock prices tomorrow

p(t) depends on p(t-1), p(t-2)....

TRAINING EXAMPLES ARE CORRELATED!!!


x: input
h: activation after applying the activation function on the output

How to train a recurrent neural network?
~we can unroll it in time in order to end up with a standard feedforward neural network:
we know how to deal with it

How to train a recurrent neural network?

~ we cna unroll it in time in order to end up with a standard feedforward neural network: we know how to deal with it

As you can see, serveral parameters are shared accross every single layer!!!

for a feed-forward network these weights are different

Vanishing/exploding gradients problem
When dealing with backpropagation we have to calculate the gradients

~ we just have to apply the chain rule several times
We multiply the weights several times: if you multiply x < 1 several times the result will get smaller and smaller

VANISHING GRADIENT PROBLEM

Backpropagation Through Time(BPTT): the same as backpropagation but these gradients/error signals will also flow backward from future time-steps to current time-steps

We multiply the weights several times: if you multiply x > 1 sereral times the result will get bigger and bigger

It is a problem when dealing with Recurrent Neural Networks usually
~because these networks are usaually deep!!!

-> why vanishing gradient is problem?
Because gradients become too small: difficult to model long-range dependencies

-> for recurrent neural networks, local optima are a much more significant problem than with feed-forward neural networks
~ error function surface is quite complex

These complex surfaces have several local optima and we want to find the global one: we can use meta-heuristic approaches as well

EXPLODING GRADIENTS PROBLEM
-> truncated BPTT algrithm: we use simple backpropagation but
We only do bckpropagation through k time-steps

-> adjust the learning rate with RMSProp(adaptive algorithm)
We normalize the gradients: using moving average over the root mean squared gradients

VANISHING GRADIENTS PROBLEM
-> initialize the weights properly(Xavier-initialization)
-> proper activation functions such as ReLU function
-> using other architectures:LSTM or GRUs