Tuesday, 9 May 2017

Cloud Computing

Working in an investment bank. I always wonder why we dont use clouds. Latest in the market.
Therefore, curiosity lead to me read about Amazon Web services.

Amazon web services really have cool stuff in there.
It provides servers to run your applications, save your code, firewalls, caching, databases
Everything that is require for running your business (hardware and Software) can be on cloud.
Cool isn't it? No it is not.

Everything on cloud, for a business like investment bank.
One can't risk it due to following reasons -
1. Vulnerable for attack
2. Downtime
3. security and privacy
4. Limited control & flexibility
A bank  can't afford first three points at any cost. As any of this can make loss of millions or even billions.

Since so much things provided in AWS, for any system in bank which doesn't directly effect business. we can definitely analyse cloud computing.

Or we can think of making such a service (cloud computing) within the bank which is reliable, secure and as vulnerable as other bank systems making its easy to manage the servers and services then making the banking system cost effective

Quick Sort

public class QuickSort {
    private final int[] arr;

    QuickSort(int[] arr) { this.arr = arr; }

    public void sort() {
        sort(0, arr.length - 1);
        System.out.println(count);
    }

    static int count = 0;

    private void sort(int p, int q) {
        count++; int pivot = partition(p, q);
        if (pivot - 1 > p) sort(p, pivot - 1);
        if (pivot + 1 < q) sort(pivot + 1, q);
    }

    private int partition(int p, int q) {
        int x = arr[p];
        int i = p + 1;
        int j = q;
        while (i <= j) {
            if (arr[i] < x) i++;
            else if (arr[j] > x) j--;
            else if (arr[i] > x && arr[j] < x) swap(i++, j--);
        }
        swap(p, i - 1);
        return i - 1;
    }

    private void swap(int i, int j) {
        int tmp = arr[i]; arr[i] = arr[j]; arr[j] = tmp;
    }
}

Caching

Here we are caching "int" "pageNumber" in cache. in queueNode.
Objects can be saved instead of "int" and Caches can be maintained and objects can be removed from Cache based on policy. that we need to define in our saved "Object".

Here I have given Java Implementation of LRUCache. Soon I will be back with another policy cache.

public class LRUCache {
    Queue queue; Hash hash;

    public LRUCache(int numberOfFrames, int capacity) {
        queue = new Queue(numberOfFrames);
        hash = new Hash(capacity);
    }

    public static void main(String[] args) {
        // Let cache can hold 4 pages & 10 different pages        // can be requested (numbered from 0 to 9)        LRUCache cache = new LRUCache(4, 10);
        cache.referencePage(1);        cache.referencePage(2);
        cache.referencePage(3);        cache.referencePage(1);
        cache.referencePage(4);        cache.referencePage(5);
        cache.queue.print();
    }

    class QNode {
        QNode prev, next;
        int pageNumber;
        QNode(int pageNumber) { this.pageNumber = pageNumber; }
    }

    class Queue {
        int count, numberOfFrames;
        QNode front, rear;
        Queue(int numberOfFrames) { this.numberOfFrames = numberOfFrames; }

        void print() {
            QNode temp = front;
            while (temp != null) {
                // Let us print cache frames after the above referenced pages                System.out.println(temp.pageNumber);  temp = temp.next;
            } } }

    class Hash {
        int capacity;
        QNode array[];

        Hash(int capacity) { this.capacity = capacity;
            array = new QNode[capacity];
        } }

    boolean areAllFramesFull(Queue queue) {
        return queue.count == queue.numberOfFrames;
    }

    boolean isQueueEmpty(Queue queue) { return queue.rear == null; }

    void deQueue(Queue queue) {
        if (isQueueEmpty(queue)) return;
        // if this is the only node in list, then change front        if (queue.front == queue.rear) queue.front = null;
        queue.rear = queue.rear.prev;
        if (queue.rear != null) queue.rear.next = null;
        queue.count--;
    }

    void enQueue(Queue queue, Hash hash, int pageNumber) {
        if (areAllFramesFull(queue)) {
            hash.array[queue.rear.pageNumber] = null;
            deQueue(queue);
        }
        QNode temp = new QNode(pageNumber);
        temp.next = queue.front;
        if (isQueueEmpty(queue)) queue.rear = queue.front = temp;
        else { queue.front.prev = temp;
            queue.front = temp;
        }
        hash.array[pageNumber] = temp;   queue.count++;
    } 
    
    void referencePage(int pageNumber) {
        QNode reqPage = hash.array[pageNumber];
        // the page is no3t in cache, bring it        if (reqPage == null) enQueue(queue, hash, pageNumber);
            // page is there and not at front, change pointer        else if (reqPage != queue.front) {
            // Unlink rquested page from its current location in queue.            reqPage.prev.next = reqPage.next;
            if (reqPage.next != null) reqPage.next.prev = reqPage.prev;
            // If the requested page is rear, then change rear as this node will be moved to front            if (reqPage == queue.rear) {
                queue.rear = reqPage.prev;
                queue.rear.next = null; }
            // Put the requested page before current front            reqPage.next = queue.front;  reqPage.prev = null;
            // Change prev of current front            reqPage.next.prev = reqPage; 
            // Change front to the requested page            queue.front = reqPage;
        } } }

Thursday, 27 April 2017

Priority Queue

Priority Queue is data structure that makes use of  Heap Data structure  and its properties (Max Heap & Min Heap)

Data structure for maintaining set S of elements, each element associated with a key.

As Heap, priority queue is also of 2 types
1. Max Priority Queue
2. Min Priority Queue

Max Priority Queue Implementation

import java.util.Arrays;

public class MaxPriorityQueue {

    int[] arr = null;
    int size;

    public MaxPriorityQueue() {
        this.arr = new int[10];
        size = 0;
    }

    public int maximum() throws Exception {
        if (isEmpty()) throw new Exception("Queue empty");
        return arr[0];
    }

    private boolean isEmpty() { return size() == 0; }

    private int size() { return size; }

    public int extractMaximum() throws Exception {
        int max = maximum();
        arr[0] = arr[--size];
        maxHeapify(0);
        return max;
    }

    public void insertKey(int key) throws Exception {
        arr[size++]=Integer.MIN_VALUE;
        heapIncreaseKey(key);
    }

    private void heapIncreaseKey(int key) throws Exception {
        if(arr[size-1]>key) throw new Exception("New Key is smaller than current key");
        arr[size-1]=key;
        int i=size-1;
        while(i>=0 && arr[parent(i)]<arr[i]){
            swap(parent(i),i);
            i=parent(i);
        }
    }

    private int parent(int i) { return (i-1)/2; }

    private void maxHeapify(int i) {
        int l = left(i);
        int r = right(i);
        int largest = i;
        if (l < size() && arr[l] > arr[largest])
            largest = l;
        if (r < size() && arr[r] > arr[largest])
            largest = r;
        if (largest != i) {
            swap(largest, i);
            maxHeapify(largest);
        }
    }

    private void swap(int largest, int i) {
        int temp = arr[largest];
        arr[largest] = arr[i];
        arr[i] = temp;
    }

    private int right(int i) { return 2 * i + 2; }

    private int left(int i) { return 2 * i + 1; }

    @Override    public String toString() {
        StringBuilder sb = new StringBuilder("");
        Arrays.stream(arr).forEach(j->sb.append(j +" "));
        return sb.toString();
    }
}

Insertion Merge Sort

Insertion sort complexity - O(cn2) where c is a constant 
Merge sort having the complexity O(anlogn where a is constant) is better than insertion sort. 

However c < a, making insertion sort perform better than merge sort when n is smaller.
Hence, many implement sorting algo as a combination of both sorts to achieve better performance. Java uses this implementation.

Implementation of InsertionSort - 
public class InsertionSort {
    public static void sort(int[] array) {
        for (int i = 1; i < array.length; i++) {
            int x = array[i];
            int j = i - 1;
            for (; j >= 0; j--) {
                if (x < array[j]) {
                    array[j + 1] = array[j];
                } else {
                    break;
                }
            }
            array[j + 1] = x;
        }
    }
}

Implementation of InsertionSort - 
public class MergeInsertionSort {
    int[] arr;

    public MergeInsertionSort(int[] arr) {
        this.arr = arr;
    }

    public void mergeSort() {
        int p = 0;
        int r = arr.length - 1;
        int q = arr.length / 2;
        mergeSort(p, q, r);
    }

    private void mergeSort(int p, int q, int r) {
        if (r - p + 1 < 8) {
            insertionSort(p, r);
        } else {
            mergeSort(p, (q - p) / 2, q - 1);
            mergeSort(q, (r - q + 1) / 2, r);
            merge(p, q, r);
        }
    }

    private void merge(int p, int q, int r) {
        int temp = p;
        int tempQ = q;
        int a[] = new int[r - p + 1];
        for (int i = 0; i < a.length; i++) {
            if (p < tempQ && arr[p] < arr[q]) {
                a[i] = arr[p++];
            } else if (q <= r) {
                a[i] = arr[q++];
            }
        }
        for (int i = 0; i < a.length; ) {
            arr[temp++] = a[i++];
        }
    }

    private void insertionSort(int p, int r) {
        for (int i = p + 1; i <= r; i++) {
            int x = arr[i];
            int j = i - 1;
            for (; j >= p; j--) {
                if (x < arr[j]) {
                    arr[j + 1] = arr[j];
                } else {
                    break;
                }
            }
            arr[j + 1] = x;
        }
    }
} 

Monday, 24 April 2017

Heap Sort

The term 'Heap' used in both the data structure and Memory leads to developers mind to find a relationship between 2 terms.

But there is no relation between the terms are totally different context. and here we are talking about heap data structure.
the description heap used for data structure is because "greater" objects are placed higher up in the tree (where "greater" is determined by an arbitrary key function) - i.e. there's a sort of piling of smaller objects on top of larger ones (or larger on top, depending how you think of it).
tree data structure. In this post, we learn about binary heap only.
The (binary) heap data structure is an array object that can be viewed as a nearly complete binary tree. Each node of the tree corresponds to an element of the array that stores value in the node. The tree is completely filled on all levels except possibly the lowest, which is filled from the left up to a point

Types of Binary Heap -
1) Max Heap - Array[parent(i)] > Array [i]
 2) MinHeap - Array[parent(i)] < Array [i]



Implementation of Heap Sort -
(MaxHeap - Complexity nlogn)
import java.util.Arrays;

public class HeapSort {


    private static int[] arr;
    private static int size;

    public static void sort(int [] arr){
        HeapSort.arr=arr;
        size = arr.length;
        Arrays.stream(arr).forEach(i->System.out.print(i +" "));
        System.out.println();                                                                                                                                                                                                                                                                                                                                                                              
        buildMaxHeap();
        for(int i=size-1;i>0;i--){
            swap(0,i);
            size--;
            maxHeapify(0);
        }
        Arrays.stream(arr).forEach(j->System.out.print(j +" "));
    }
//Since nearly complete binary tree have n/2 leaves 
//and leaves being the only element dont need to be heapify
//therefore we start from n/2 elements till the 1st element and create the max heap 
    private static void buildMaxHeap() {
        for (int i=size/2;i>=0;i--
             ) {
            maxHeapify(i);
        }
    }

// Max heapify the ith element of the array 
//i.e. A[i] is the largest value of for all elements succeeders of i
// complexity nlogn
    private static void maxHeapify( int i) {
        int l = left(i);
        int r = right(i);
        int largest = i;
        if(l<size && arr[l]>arr[i])
            largest = l;
        if(r<size && arr[r]>arr[largest])
            largest = r;
        if(largest!=i) {
            swap(largest, i);
            maxHeapify(largest);
        }

    }

    private static void swap( int i, int i1) {
        int temp = arr[i];
        arr[i]=arr[i1];
        arr[i1]=temp;
    }


    static int parent(int [] arr, int i){
        return (i-1)/2;
    }

    static int left( int i){
        return 2*i+1;
    }

    static int right( int i){
        return 2*i+2;
    }


}

Sunday, 23 April 2017

Garbage Collection

 A system that automatically recycles unused memory in a programming language.

Pros -
  • Makes programming easier
  • Helps developers to avoid several memory problems
Cons -
  • Consumes computing resources
  • Consumes memory to store status of objects
  • Recycle process may even block the program potentially
Memory Problems -
  • Dangling pointers - point to an object that no longer exists
  • Freeing a region of memory that s already free
  • Memory leaks - Unreachable region of memory that can never be freed
Designing Garbage Collection -    All objects in-memory heap are like a directional graph. All the nodes(vertices) which are not reachable from root(s) can be freed.
Problem - Find unreachable nodes in a directional graph
Solution -

Edge.java
public class Edge<T> {
    private final int cost;
    private final Vertex<T> to;
    private final Vertex<T> from;
    public Edge(Vertex<T> from, Vertex<T> to, int cost) {
        this.cost = cost;
        this.to = to;
        this.from = from;
    }
    public Vertex<T> getTo() { return to; }
    public Vertex<T> getFrom() { return from; }

    @Override   public boolean equals(Object obj) {
        Objects.requireNonNull(obj);
        if(this==obj) return  true;
        Edge other = null;
        if(obj instanceof Edge)
            other = (Edge)obj;
        if (!this.getFrom().equals(other.getFrom())) return false;
        if (!this.getTo().equals(other.getTo()))  return false;
        return true;
    }
}

Vertex.java 
public class Vertex<T> {

    private Color markState;
    private final String name;
    private final T value;
    private List<Edge<T>> edges;
    private List<Edge<T>> incomingEdges;
    private List<Edge<T>> outgoingEdges;

    public Vertex(String name, T value) {
        this.name = name;
        this.value = value;
        incomingEdges = new ArrayList<>();
        outgoingEdges = new ArrayList<>();
        edges = new ArrayList<>();
    }

    public Edge<T> findEdge(com.graph.Vertex<T> to) {
        Optional<Edge<T>> e = edges.stream()
        .filter(edge -> edge.getFrom().equals(this) && edge.getTo().equals(to))
        .findAny();
        return e.isPresent() ? e.get():null;
    }

    public void addEdge(Edge<T> e, boolean isIncoming) {
        this.edges.add(e);
        if (isIncoming) { incomingEdges.add(e);
        } else {  outgoingEdges.add(e); }
    }

    public void remove(Edge<T> e) {
        incomingEdges.remove(e);
        outgoingEdges.remove(e);
        edges.remove(e);
    }

    public void clearMark() { this.markState = null; }

    public void setMarkState(Color markState) {
        this.markState = markState;
    }

    public Color getMarkState() { return markState; }

    public String getName() { return name; }

    public T getValue() { return  value; }

    public List<Edge<T>> getOutgoingEdges() { return outgoingEdges; }

    public List<Edge<T>> getIncomingEdges() { return incomingEdges; }
}
Color.java
public enum Color { BLACK, WHITE, GREY }
Graph.java
class Graph<T> {
    Set<Vertex<T>> vertices;
    List<Edge<T>> edges;
    Vertex rootVertex;

    public Graph() {
        vertices = new HashSet<>();
        edges = new ArrayList<>();
    }

    public boolean isEmpty() {  return size() == 0; }
   
    public Vertex getRootVertex() { return rootVertex; }

    public void setRootVertex(Vertex rootVertex) {
        this.rootVertex = rootVertex;
        this.addVertex(rootVertex);
    }

    public Vertex getVertex(T value) {
        return vertices.stream().filter(val -> value.equals(val)).findAny().get();
    }

    public boolean addEdge(Vertex<T> from, Vertex<T> to, int cost) {
        if (!(vertices.contains(from) || vertices.contains(to))) {
            throw new NoSuchElementException("Element not found");
        }
        Edge<T> e = new Edge<T>(from, to, cost);
        if (from.findEdge(to) == null) {
            from.addEdge(e, false);
            to.addEdge(e, true);
            edges.add(e);
            return true;
        }
        return false;
    }

    private int size() { return vertices.size(); }

    public boolean addVertex(Vertex<T> vertex) {
        Objects.requireNonNull(vertices, "Vertices are null");
        return vertices.add(vertex);
    }

    public boolean insertBiEdge(Vertex<T> from, Vertex<T> to, int cost)
            throws IllegalArgumentException {
        return addEdge(from, to, cost) && addEdge(to, from, cost);
    }

    public Vertex<T> findVertexByName(String name) {
        return vertices.stream().filter(vertex -> vertex.getName().equals(name))
             .findAny().get();
    }

    public Vertex<T> findVertexByValue(T value) {
        return vertices.stream().filter(vertex -> value.equals(vertex.getValue()))
              .findAny().get();
    }

    public String toString() {
        StringBuffer tmp = new StringBuffer("Graph[");
        vertices.stream().forEach(v -> tmp.append(v.getValue()).append(" "));
        tmp.append(']');
        return tmp.toString();
    }

    public boolean removeVertex(Vertex<T> v) {
        if (v == rootVertex)
            rootVertex = null;
        // Remove the edges associated with v     
            v.getOutgoingEdges().forEach(e -> {
            v.remove(e);
            Vertex<T> to = e.getTo();
            to.remove(e);
            edges.remove(e);
        });

        v.getIncomingEdges().forEach(e -> {
            v.remove(e);
            Vertex<T> from = e.getFrom();
            from.remove(e);
            edges.remove(e);
        });
        return true;
    }

    public boolean removeEdge(Vertex<T> from, Vertex<T> to) {
        Edge<T> e = from.findEdge(to);
        if (e != null) {
            from.remove(e);
            to.remove(e);
            edges.remove(e);
            return true;
        }
        return false;
    }
    public void clearMark() {
        vertices.stream().forEach(vertex -> vertex.clearMark());
    }
    public void garbageCollection() {
        vertices.stream().forEach(v -> v.setMarkState(Color.WHITE));
        visit(getRootVertex());
        vertices.stream().filter(vertex -> vertex.getMarkState() == Color.WHITE)
              .forEach(vertex -> removeVertex(vertex));
        vertices.removeIf(vertex -> vertex.getMarkState() == Color.WHITE);
        clearMark();    }
    private void visit(Vertex<T> v) {
        v.setMarkState(Color.GREY);
        v.getOutgoingEdges().stream().forEach(edge -> {
            Vertex<T> u = edge.getTo();
            if (u.getMarkState() == Color.WHITE)
                visit(u);
            v.setMarkState(Color.BLACK);  });
    }
Main class - DirectionalGraph.java
public class DirectionalGraph { public static void main(String[] args) { Graph graph = new Graph(); Vertex<String> vertexA = new Vertex<>("a", "a"); Vertex<String> vertexB = new Vertex<>("b", "b"); Vertex<String> vertexC = new Vertex<>("c", "c"); Vertex<String> vertexD = new Vertex<>("d", "d"); Vertex<String> vertexE = new Vertex<>("e", "e"); Vertex<String> vertexF = new Vertex<>("f", "f"); graph.setRootVertex(vertexA); graph.addVertex(vertexA); graph.addVertex(vertexB); graph.addVertex(vertexC); graph.addVertex(vertexD); graph.addVertex(vertexE); graph.addVertex(vertexF); graph.addEdge(vertexA, vertexB, 5); graph.addEdge(vertexA, vertexC, 5); graph.addEdge(vertexB, vertexD, 5); graph.addEdge(vertexB, vertexE, 5); graph.addEdge(vertexD, vertexE, 5); System.out.println(graph); graph.garbageCollection(); System.out.println(graph); } }
Output - 

Graph[e d b a c f ]
Graph[e d b a c ]
  1. As f is not connected to any node is removed from Graph. Garbage collected