Data Warehousing | DWH » Basic Concepts in Data Warehousing

Star Schema in Data Warehouse

Admin — Tue, 02 Feb 2010 12:06:22 +0000

Star schema contain a single or mutiple fact table surrounded by dimension table.The dimension table in star schema will be denormalized.Most of the datawarehouse are designed according to star schema.Accordingly star schema as following features

1. Single(Simple Star Schema) or multiple(Complex Star Schema) fact table
2. There are fewer number of dimension table hence lesser joines and queries execute faster
3. Considered a special case of snowflake schema.
4. Dimensional tables are generally in 2nd normal form while fact table are in third normal form

grep command in unix with example

Admin — Fri, 15 Jan 2010 08:24:44 +0000

Grep command is very useful in UNIX.Grep searches for similar pattern on file and return the result.Example of grep command is
grep ‘word’ filename – Will return the line which contain word.
grep -i ‘nocase’ filename – This option will make grep case insensitive.
grep – r ‘sear’ /etc/ – This will search in the whole directory recursively
grep “boo” filename – This will display line which contain exact match of the word boo.
egrep -w ‘word1|word2′ filename – This will search for both the word1 and word2 in the filename.
grep – c word1 filename – Will return the number of lines where word1 is found.
grep -v word1 filename – Will return the line which doesn’t contain the word1.
grep word1 * – Will search for occurence of word1 in whole directory
grep ^a.ad fruitlist.txt – Print line that contain the letter followed by a and in last contain letter ad.

File Compression and archival in UNIX

Admin — Fri, 15 Jan 2010 07:23:56 +0000

There are lot of times when you need to compress and archive files in UNIX in data warehousing project.The reason for this is files in DWH are huge.In this article I will explain how to compress files and than archive them.Files compression is not useful for small files.There are few command and utilities you can use to compress files.They are as follows

1. compress
2. gzip
3. tar(with c option) – Arcchives files without compression
Archive may contain one or more files.Generally archival is done for files which are not used much.

The command to decompress are as follows respectively
1.uncompress
2. gunzip
3. tar(with x option)

Snowflake Schema in data warehouse

Admin — Mon, 11 Jan 2010 06:57:52 +0000

A Snowflake schema has a centralized fact table surrounded by dimension tables.The dimension table in snow flake schema are normalized.There will be single fact table joined to multiple dimension table which in turn may be joined to other dimension tables.So snowflake schema has following features

1. Large number of dimension tables.
2. Since dimension table are normalized the size of dimension table is small
3. For query purpose we need to retrieve data from multiple dimension table resulting in more joins and slower performance.

ETL tools in data warehousing?

Admin — Fri, 08 Jan 2010 07:16:25 +0000

Once you have decided to build a data warehouse for the company.The challenges lies in deciding which ETL tool to use.There are plenty of ETL tools like Pentaho Data Integration,Abnitio.The most used ETL tool is Informatica.ETL stands for Extract,Transform and Load.Although most of ETL tool can perform all the three function.Generally one or more process of extraction,transformation and load may be performed by some external software.Lets us understand all the three process one by one

Extract – The process involves extraction fo data from homogenoes or heterogenoes sources to load into the data warehouse.

Transform – Once the data is extracted various transformation need to be performed so that the data is useful for various analysis.

Loading – The last step is to load the the data into datawarehouse.

The list of popular ETL tools in the market are
1. Informatica
2. Oracle Warehouse Builder
3. Pentaho
4. Ab Initio
5. Datastage

What is Slowly changing Dimension?

Admin — Sun, 03 Jan 2010 06:34:36 +0000

A slowly changing dimension is dimension table in which attribute changes with time.Suppose we have a customer who has bank account in Chicago.Now he changes his location and moves to new york.A dimension table for this customer will typically contain the following information

Name	Age	City	Gender
Jack	25	Chicago	Male

Now the challenge is to how to store this changing information.In a datawarehouse this is generally done in three ways.Accordingly such type of dimension are called type 1,type 2 and type 3

Type 1 Dimension
Type 1 dimension will not maintain history of the data at all.In the following example if the customer moves from one city to another.The new information will overwrite the old information.

Type 2 Dimension
As you remember in my previous article I explained we have surrogate key as primary key for dimension table.Now in type 2 dimension we maintain all the history.Suppose Jack is customer of bank he moves from Chicago to Miami and than to New york.The table will contain all this information in record form.Each time a new surrogate key is for created for each record.

Type 3 Dimension
Type 3 dimension will generally store limited history value.In most cases it will store the current and previous value..Suppose Jack is customer of bank he moves from Chicago to Miami and than to New york.Table will store Miami and New York.The information for Chicago will be deleted.

What is Dimension Table?

Admin — Sun, 03 Jan 2010 06:11:38 +0000

Dimension table contain attribute which describe fact table.In my previous article I explained fact is nothing but measures to value business like sales.Dimension on the other hand are used to analyze fact.A example of dimension table will be

Region

CountryStateCity

Now Fact table will contain some column like sales.So dimension table can be used to analyze sales according to Country like USA,UK.Dimension table generally have surrogate key as primary key.The size of dimension table is small.Unlike fact table which grows in size dimension table remain mostly static.Referential integrity must be maintained between dimension table and fact table.
Dimension table will generally contain hierarchies.Hierarchy help in analyzing the data.For example in the table the hierarchy is Country,State and City.This help in knowing the sales in particular city,state or country.Mainly dimensional table will contain textual information which make sense like country.year,etc.

Fact Table in Datawarehouse

Admin — Thu, 31 Dec 2009 11:22:49 +0000

Fact table is the centre table in snow flake scheme.A fact is measure of business for example sales.Generally facts are numbers.Dimension on other help us to analyze facts.Example of Dimension will be Region.Apart from these measures the fact table contain foreign key for dimension table.Fact table is generally large in size.Fact table are highly normalized because of that reason.

Measures in Fact Table can be of three type

Additive: Additive facts are facts that can be summed up through all of the dimensions in the fact table.
Semi-Additive: Semi-additive facts are facts that can be summed up for some of the dimensions in the fact table but not the others.
Non-Additive: Non-additive facts are facts that cannot be summed up for any of the dimensions present in the fact table.

Let us see all three type of measures in fact table.Suppose you have fact table has

Region
Year
Per-Profit
Sale

Sale and Per-Profit are the facts while region & Year are dimension.Sale is additive fact since it makes sense to add it across Region and Year.Percentage profit is non-additive fact as it doesn’t make sense to add it across Region and Year.

Every fact table has primary key which help in uniquely identifying the record.
Some fact table don’t contain facts at all.Such table are called factless Fact table.These table are used to find event which occurred or didn’t occurred.

The other concept of fact table is to understand what is meant by granularity.Granularity means what is the lowest level of information that will be stored in fact table.