Databases and Business Intelligence - Data Mining Tools - Assessment Answer

Databases and Business Intelligence Assignment

Assignment Task

Part 1

There are many large datasets available at:

https://data.london.gov.uk/

https://www.microstrategy.com/cloud/personal/datasets/

Choose TWO large datasets and analyse them with pivot tables. Document the insights and trends that you find during the analysis. Submit a word document that includes FOR EACH DATASET:

1. The URL of the dataset.
2. A description of the dataset.
3. A screen capture showing the first page of the Excel spreadsheet containing the dataset.
4. Screen captures of TWO DIFFERENT pivot tables on the dataset utilised together with anygraphs output.
5. A clear analysis of your findings.
6. Focus on the insight you are trying to gain: try differentiating dimensions from facts in the dataset. You will usually have dimensions as the rows (time, location, product type) and facts in the centre (revenue, cost etc).

Part 2

Write a 1200 word (I am not going to count them) technical report (in MS Word), complete with proper referencing, from the position of a professional business analyst, to address the following:

(a) Discuss the important features of data mining tools; and

(b) Discuss how data mining can realize the value of a data warehouse.

This exercise is from questions 18.13 and 18.14 on page 460 of the textbook.

Part 3

The four “V”s of big data are volume, variety, velocity and veracity which reflect the amount of data, the different types, the speed with which it is collected, and the uncertainty relating to its truth.

You are a large department store thinking about using big data to understand your customers better.

Draw an entity relationship model containing key attributes from a customer’s internet browsing activity, your transaction sales database, social media activity and publicly available demographic data on your site (or any other interesting sources – CCTV, telephone). The title of the diagram should contain the purpose of the model, what you are trying to achieve.

Submission of assignment You can put both parts 1 and 2 in ONE Word document and email it to me from an ECU email address. I will acknowledge receipt by email. If you do not receive acknowledgement then it means that I have not received it.

Part: 1

URL of the dataset

https://data.london.gov.uk/dataset/granted-british-citizenship

Description of the dataset

The dataset shows the total number of person attending a British Citizenship Ceremony. (P) Provisional figures.(R) Figures have been revised since the first release of this data, for example to include late returns. The dataset relates to adults over 18 years of age only.

Ceremony Attended: A ceremony organised by County or Local Authorities for successful applicants over 18 years of age for British citizenship. At the ceremony the applicant takes the Oath or Affirmation of allegiance to Her Majesty the Queen and the Pledge of loyalty to the United Kingdom. Since 1 January 2004 this has been the final stage in the process of attaining British citizenship.

Screen captured of 1st page of the dataset

Clear analysis of the finding and focusing on the insights

The dataset shows the people who attended British citizenship ceremony. From the two pivot tables the above graph can be found. It shows the total number of people granted by different authorities from the year 2010 to 2015.

The graph shows that London Newham granted citizenship to the most number of people in 2013. The lowest number of people granted citizenship by London Corporation of the city of London in 2015 and the total number was only 17.

From the above analysis it is clear that London Brent and London Newham grants the most number of people citizenship every year. On the other hand London Corporation of the city of London and London Havering grant citizenship to the lowest number of people.

Part: 2

Introduction

Data mining is a tool, technique or practice to search large datasets automatically with a view to identify trends and patterns. Data mining tools are used to predict and calculate different types of series of data to understand the future occasions. Basically data mining tools help to discover examples, forecast results, produce noteworthy data and also analyse large datasets easily. (Benson, 2012)

Important Features of Data Mining Tools

Data mining has many features, which gives the advantage in case of analysing large datasets and databases. They help to answer questions that cannot be addressed easily. They observe past trends and forecast future ones. Among the features of data mining tools, there are some important ones that are notable. They are discussed below. (Thakur, 2015)

Data preparation

Data preparation features are provided by data mining tools at the time of analysing large datasets. It is in this connection implies control of information into a shape appropriate for further investigation and handling. It is a procedure that includes a wide range of assignments and which can't be completely mechanized. A considerable lot of the information readiness exercises are normal, repetitive, and tedious. It has been evaluated that information arrangement represents 60%-80% of the time spent on an information mining venture. (Website, 2016)

The most time consuming part in data mining is the data preparation stage. The process of data preparation is provided in different ways and in different speed in different data mining tools. Data cleansing is one of the most complicated part that a data mining tool provide and it greatly supports the data preparation stage. In data cleansing, unnecessary data is removed from the database and it makes the dataset easily analysable. Besides that, it can handle missing data and describe each of the data properly. It can also transform data and sample data to validate datasets for speeding up the analysing process.

Selection of data mining algorithms

In order to meet the requirements of the user while performing data analysis, it is very important to find out the most appropriate data mining algorithm so that the analysis can be done properly. There are different type of algorithms that are being used all over the world. Every algorithm has different purpose. To select the proper data mining algorithm, different aspects needs to be identified. They are, understanding how the data types will be treated by the algorithms, how the predictor values are responding with the algorithm, how fast the algorithm can train and how quickly it can work with the new data.

Noise is another very important thing to consider in case of algorithm. The sensitivity of the algorithm to the noise determines how useful the algorithm will be. Basically, noise is the value that is used to differentiate between model created by the algorithm and the predictions. The data can be identified as noisy if there are too much errors including missing values, incorrect data etc.

Product scalability and performances

Performance and scalability are very important features of a data mining tools as they help to identify how capable the data mining tool is in case of dealing with huge amount of data that are frequently increasing. The sophisticated validation control of the data sets done by the data mining tools also indicates its scalability and performance. If the tool can provide enough support to be scalable and also show performance which is on a satisfactory level, then it can be said that the data mining tool has what it takes to analyse large datasets properly.

Facilities for understanding results

One of the most important features of a data mining tool is to make the user understand the results. Using the data mining tools, users can easily identify trends and patterns so that they can understand the results properly. Besides that, different visual graphics can help the users to properly understand the data without any difficulties.

How Data Mining Realizes the Value of a Data Warehouse

Data mining is an important technique in case of analysing large datasets. In today’s world, the data is increasing every day and it has become very difficult to store and analyse the data. For make things easier, data warehouse is being used. Data mining can be easily done within a data warehouse as it realizes the value of a data warehouse.

Data mining tools require data and information with good consistency and good quality and a data warehouse can provide such kind of data in a large quantity. Thus, data mining tools understand the value of a data warehouse. Data warehouse collects data from different sources and data mining tools can easily analyse them. To get the best structured data, data warehouse is required.

Data warehouse automation is a truly well-kept mystery. Consider it a contemporary way to deal with building and overseeing information distribution centers of all shapes and sizes. This incorporates business-confronting information stores and information vaults, autonomous of design. The customary techniques for working out that information framework, where we demonstrate the information, use the ETL instruments, and inspire source to target maps – the majority of that waterfall system – has turned out to be tricky with regards to responsiveness to the business. The business now is restless for information in a shape they need in a time allotment that they require. Automation empowers associations to significantly accelerate the capacity to make that foundation. (Eckerson, 2016)

Automation does things locally in the information stockroom target stage. We make information definition dialect (DDL) to make tables. It makes information control dialect (DML) as put away systems in the local database to populate the tables. Also, maybe most strikingly, computerization makes the documentation no matter how you look at it – something that is once in a while done well and some of the time not done by any means. Mechanization innovation ensures that the whole environment is all around reported. (Eckerson, 2016)

Information warehousing has been an imperative part of the information design and IT framework of numerous associations for very nearly 30 years. Be that as it may, in spite of its legacy, disarray has blockaded the information stockroom lately. (Eckerson, 2016)

With the coming of information lakes, huge information and progressed investigation, some inside the IT business have addressed whether the information distribution center is still pertinent. The short answer is: totally. The more drawn out answer is tended to in this four-section arrangement, which points of interest what you have to know before purchasing an information stockroom stage. (TDWI, 2016)

Notwithstanding all the buildup encompassing huge information and examination, organizations in all businesses and of everything except the littlest size are utilizing information stockrooms to convey significant BI that administrators and chiefs can use to settle on critical choices. Information distribution center stages give representatives a window into their association's verifiable exercises - both victories and disappointments. While huge information and examination do have their place, information warehousing emerges as a commonsense, demonstrated practice for investigating organized business information with a specific end goal to bolster information driven basic leadership. (TDWI, 2016)

Conclusion

Data mining tools are very helpful to analyse large datasets and find out different answered to difficult question that cannot be answered with normal database tools. As the increase of data warehouse shows a good sign, data mining tools can be easily implemented within a data warehouse so that the stored data can be processed and analysed properly.

Part: 3

Entity Relationship Model of Customers Liked Products from Social Media