- Cloud Launcher for web app deployment

- choose machine sizes and disk sizes

- storage options: cloud buckets, persistent disk, local SSD

- storage technologies: Cloud SQL (mySQL, PostgreSQL; NoSQL)

- Load Balancing at any stack level by GCP or 3rd party products

- DevOps tools

Which of the following is a PAAS option for hosting web apps?

• Compute Engine VM
• Container Engine instance
• Cloud storage with Firebase hosting
• App Engine standard or flexible environment

Which of the following is a IAAS option for hosting web apps on GCP?

• App Engine standard environment
• Container Engine instance
• Compute Engine instance
• Cloud storage with Firebase hosting

Rank the following storage options from most expensive to cheapest (per GB)

• Cloud storage > SSD persistent disks > Local SSD > standard persistent
• Local SSD > SSD persistent disks > standard persistent > Cloud storage
• Local SSD > standard persistent disks > standard SSD > Cloud storage
• SSD (any type) > Cloud Storage > standard persistent

Local SSD > SSD persistent disks > standard persistent > Cloud storage

Rank the following options in scope of access

• Cloud storage - global, persistent (SSD and standard ) - zonal, local SSD - instance
• Cloud storage - regional, persistent (SSD and standard ) - regional, local SSD - instance
• All storage options offer global access (but billing rates vary)
• Cloud storage - global, persistent (SSD and standard ) - regional, local SSD - zonal

Cloud storage - global, persistent (SSD and standard ) - zonal, local SSD - instance

How do storage options differ with container engine instances relative to compute engine instances?

• Cloud storage is global access for container engine instances; regional for compute engine VMs
• No difference in storage options
• BigQuery and BigTable can be used from containers but not from raw compute engine
• VMs Container disks are ephemeral by default; need to use a specific abstraction to make them persistent

Container disks are ephemeral by default; need to use a specific abstraction to make them persistent

• Persistent Disks
  • Standard
  • SSDs
• Local SSD

• Multi-regional - for frequent access globally
• Regional - for frequent access regionally
• Nearline - access once a month max
• Coldline - access once a year

• Hadoop - HDFS
• GCP - Cloud storage

HDFS requires a name node.  GCP Cloud Storage does not.

• Hadoop - Hive
• GCP - BigQuery

BigQuery is far faster than Hive.  It uses columner storage. Hive is on top fo HDFS.

• Cloud SQL
• Cloud Spanner

• Cloud SQL
• Cloud Spanner

• Cloud SQL supports these
  • MySQL
  • Postgres
• Cloud Spanner is proprietary
• Cloud Spanner supports auto horizontal scaling.

DataStore

• Multi-key value store
• Very fast hash based indexes
• Very fast lookups of non-sequential keys
• Same time for queries regardless of data size; query performence depends on result size
• best for low write/high read-intensive needs
• Offers transaction support

• Cloud Storage for Firebase: compute, block storage with mobile SDK access
• Firebase Realtime DB: fast, random access with mobile SDK access.

• When transferring from another cloud provider
• When copying files from on-premise for the first time.

• Hotspotting
• Interleaving
• Splits
• Primary indexes required; Secondary Indices available (Not a feature in HBase)
• Index directives (force a query to use an index)
• STORING clause: force a column in an index
• Non-normal data types: arrays; arrays of arrays; Structs in queries only.
• Stronger than ACID: guarantees order of commitment
• Transaction modes: locking read-write; read-only; single read call -- doesn't use locking.
• Staleness timestamp bounds: Latest; bounded; exact

• Scalability
• Low admin burden
• Cluster resize without downtime
• Many more column families before performance drops

Are these supported in BigTable

• Multiple table operations?
• Indexes
• Constraints
• Grouping
• Joins
• Aggregates
• CRUD operations

• Row ID
• Column family
• Column
• Timestamp

1. Field Promotion: Reverse URL order
2. Salting: Hash the key value

It observes read and write patterns and redistributes data among shards.

• Poor schema design
• workload too small (<300 gb)
• used in short bursts
• cluster too small
• cluster just started
• using HDD instead of SSD

• Lookup by row key
• sequential scans

Which of these affects response time in datastore?

• Database size
• Resultset size

It is called the Perfect index, and the index is chosen in this order:

1. Choose equality indexes
2. If no equality condition, choose an inequality index (only one inequality condition is allowed in a search)
3. Choose an index and satisfies a sort condition.

• Restrictions
  • No Joins
  • Only one inequality condition allowed
  • Cannot filter based on subquery results
• Limitations
  • Updates are slow
  • limited ACID support

• Strong consistency
• Eventual consistency

• At creation time
• During initial load.

• Batch loads
• Streaming loads

• CSV
• JSON
• Avro
• Cloud DataStore backups

• Interactive queries
• Batch queries - which BigQuery will run when resources allow within 24 hours.
• Views
  • Authorized view for security
  • Row level permissions available
  • Can't export data from a view
  • can't mix standard and legacy SQL
  • No functions or wildcards may be used
  • Limit of 1000 views
• Partitioned tables
  • Automatically created based on load datetime
  • Automatic discarding

• Directed-acyclic graph: DAG
• Pipeline: A single Data Flow job
• PCollection: The data sets that are inputs and outputs form tranforms
• Transform: transforms the data
• Source, Sink - source and destinations
• Driver:  Defines the computation DAG (pipeline)
• Runner: executes the DAG on the backend
• Backends suppoted:
  • Apache Spark
  • Apache Flink
  • Google Cloud Dataflow
  • Beam Model

• Publisher
• Subscriber
• Messages
• Queues: one per subscription
• Acknowledgement
• Planes
  • Data plane: moves messages between publishers and subscribers. Servers here are forwarders.
  • Control plane: handles assignment of publishes and subscribers to servers on the data plane. Servers here are routers.

• Push subscriptions use a WebHook endpoint
• Pull subscriptions use an HTTPS request to an endpoint

Define the pub sub:

• Sliding window
• Sliding interval

• Sliding window:  the window of time from which all data is gathered for processing.
• Sliding interval: The amount of time a window will shift for processing the next sliding window 

They figure out by themselves what features to pay attention to.  These are commonly called "Deep Learning" systems, which generally refer to Neural Networks.

A neural network model is made up of layers that feed one another.  Each layer is composed of neurons.  The outer layers -- the input and outer layers - are called the "visible layers".  Other layers are the "hidden" layers.  It is the depth of the layers that give Deep Learning its name.

• Rank - The number of dimensions
  • 0 - Scalar
  • 1 - Vector
  • 2 - Matrix
  • >2 - n-dimensional
• Shape - The number of elements in each dimension
• Data Type

What are, in Tensorflow

• Constants
• Placeholders
• Variables

• Constants: Immutable values which do no change
• Placeholders: Assigned once and do not change after
• Variables: are constantly recomputed

In Tensorflow, what are:

• Coordinators
• QueueRunners

• Coordinators - a class to manage and work with multiple threads.
• QueueRunner - allows you to work with multiple elements from a queue in parallel using multiple threads.

• Euclidean: sqrt(sum((xj-xij)^2)), as the crow flies
• L1/Manhattan/Snake/City block: absolute values of number of horizontal and vertical steps summed

K-nearest neighbor is a machine learning algorithm for classification and regression.  It uses distance measures to find the closest matching class -- the most similar -- for a given input

• A linear (affine) transformation
  • Applies Wx + b to the intput
    • W: Weight
    • b: bias
      • These are variables finalized by the training process
• An activation function

• Rectified Linear Unit
• A very commonly used activiation function in a neural network neuron.
• Returns the max of the result from the Affine Transformation, or zero.

The process is assigned a gradient descent method which determines the learning rate by which the model travels through epochs until completion.

• Stochastic: Uses only one training sample for each epoch
• Mini-batch: Uses a subset of the whole training data set
• Batch: Uses the whole training set for each epoch

• Batch size
• Number of steps
• optimizer function

• Provides probability of a y value given an x value
• produces an X curve
• X variable can be continuous, but y variables can only be categorical, .e.g, binary
• p(yi) = 1/(1+e^-(A+Bxi))
• Often used for linear classification

What Affinity function is used with

• Linear regresssion
• Logistic regression

• Linear regresssion: identity (does nothing)
• Logistic regression: Softmax (transforms Affine transformation into probabilities