Course - Architecting Big Data Solutions

#bigdata 這東東，有點似幻似雲，學了後覺得有點好像是其實又不是。我想這個真的還是需要再深入一點才會知道怎麼回事。這個課程選的有點太跳了，畢竟沒接觸過big data 的我們突然就跳到去學習實行該注意的真的是錯了。不過經過了上週的Google Cloud seminar + 這課程，確實是受矚目的課題。蠻有趣的話題吧。


以下又是官方式報告。

The course key topics is regarding the strategies to architect Big Data. So throughout the course, there are few use job scope & concerns shared for each module.

Overview
Big data is high volume (Tera or Peta bytes) of data that enabled enhanced decision making, and process automation.

The difference between Traditional Data Solution vs Big Data Solution:

Traditional Data Solution	Big Data Solution
Raw data provided by end "users"	Data Driven Management enable advance analytics, sources from logs, social media etc.
With fixed and well defined schema, pre-defined linking, fixed data attributes	Flexible schema
Singapore centralized data repository & data store	Cloud adaption & distributed
Data moves to application code for processing	Real time streaming
pre-customized reporting, pre-summarized & computed data	Advanced analytics for reporting solutions - analytics, machine learning, predictions.
3-tier architectures - UI, business, data	Data Centered, integration oriented
High cost for storing data	Cloud focused - pay as you used
Home growth	Support open integrations

When Big Data is good, why not every organization making it happen?

1. Too many competitors - to come out with a product is extremely easy, but to have the product outstanding is too challenging and require a long run energy.
2. Stop from cloud options - organizations is not ready for a massive data volume, when it is, cloud option should be considered, but in traditional arrangement, most of the company still rely on home maintenance rather than a 3rdparty cloud service.
3. Open integration & API - before moving forward to Big Data, there should be an environment that ready to accept huge data traffic, open environment that allows public connection and communication. This is one of the product's decision that may moving forward to a different direction from current target.

7 Modules of Big Data introduced in the course.


How to start?

1. Start from a proof-of-concepts project that either help an organizations to have a clearer picture of a process OR take advantage of existing business knowledge to explore the Big Data solutions .
2. Build Sandbox and introduce to to production gradually