Skip to main content

Star schemas and aggregate (or summary) fact tables


Aggregate tables can further improve query performance by reducing the number of rows over which higher-level metrics must be aggregated. 
However, the use of aggregate tables with dimension tables is not a valid physical modeling strategy. Whenever aggregation is performed over fact data, it is a general requirement that tables joined to the fact table must be at the same attribute level or at a higher level. If the auxiliary table is at a lower level, fact rows will be replicated prior to aggregation and this will result in inflated metric values (also known as "multiple counting").

With the above Time dimension table, a fact table at the level of Day functions correctly because there is exactly one row in DIM_TIME for each day. To aggregate the facts to the level of Quarter, it is valid to join the fact table to the dimension table and group by the quarter ID from the dimension table.

Sql
select DT.QUARTER_ID,
   max(DT.QUARTER_DESC) Quarter_Desc
   sum(FT.REVENUE) Revenue
from DAY_FACT_TABLE FT
   join DIM_TIME DT
      on (FT.DAY_ID = DT.DAY_ID)
group by DT.QUARTER_ID

If, however, there is an aggregate fact table already at the level of Quarter, the results will not be correct. This is because the query must join on Quarter ID, but the quarter ID is not a unique key of the dimension table. Because any given quarter of a year contains 90, 91 or 92 days, the dimension table will contain that many rows with the same quarter ID. Thus fact rows will be replicated prior to taking the sum, and the sum will be too high.

Sql
select FT.QUARTER_ID,
   max(DT.QUARTER_DESC) Quarter_Desc
   sum(FT.REVENUE) Revenue
from QTR_FACT_TABLE FT
   join DIM_TIME DT
      on (FT.QUARTER_ID = DT.QUARTER_ID)
group by FT.QUARTER_ID

This is a generally recognized problem with star schemas, and is not strictly a MicroStrategy limitation.

Star schemas will function correctly with MicroStrategy SQL Generation Engine 8.x as long as they obey the general data warehousing principle that fact tables should not be at a higher level than the dimension tables to which they are joined.

If aggregate tables are required, it is necessary to provide higher-level lookup tables with unique rows corresponding to each aggregate table's key. Logical views are a way to do this without adding tables or views to the warehouse. For example, LWV_LU_QUARTER may be defined using the following SQL statement:

Sql
select distinct QUARTER_ID, QUARTER_DESC, YEAR_ID
from DIM_TIME

 

With this logical view, it becomes possible for MicroStrategy SQL Generation Engine 8.x to query the quarter-level fact table as follows. Since the logical view has distinct rows per quarter, multiple counting will not occur in this query.

Sql
select FT.QUARTER_ID,
   max(LQ.QUARTER_DESC) Quarter_Desc
   sum(FT.REVENUE) Revenue
from QTR_FACT_TABLE FT
   join (select distinct QUARTER_ID, QUARTER_DESC, YEAR_ID
         from DIM_TIME) LQ
      on (FT.QUARTER_ID = LQ.QUARTER_ID)
group by FT.QUARTER_ID

For more information on the use of logical views in MicroStrategy SQL Generation Engine 8.1.x and 9.x, consult the MicroStrategy Project Design Guide manual, Appendix B: Logical Tables, "Creating logical tables."

Aggregate tables store pre-summarized totals at a higher level of aggregation than the most granular fact table. They allow reports to be generated from small, rather than large, tables; therefore, performance is enhanced. A successful aggregation strategy seeks to choose aggregate tables that will have the most impact while taking the least amount of space.

Aggregation decisions are driven by the following factors:

  • Usage patterns: Build aggregate tables that are likely to be used the most.
  • Compression ratios: The compression ratio between two tables is defined as the size of the aggregate compared to the size of the smallest table from which the aggregate can be derived.
  • Volatility: Changes in hierarchies over time impact the accuracy of aggregate tables. Sometimes aggregate tables must be rebuilt as a result of changes in dimensions.
A good candidate for aggregation should have at most 10-15 percent of the size of the smallest table from which it is derived.

EXAMPLE:
The MicroStrategy Tutorial project uses aggregate tables by default. A simple metric sum (Revenue) will go to different aggregate tables depending on the attributes on the template.

  1. Create a report with Year on the rows and Revenue on the columns.
  2. Execute the report and view the SQL:

  3. Drill from Year to Item and view the SQL:

The query will go from using ORDER_FACT to ORDER_DETAIL. When Year is on the template, the engine selects the smaller table and the fact is calculated as:

sum(a11.ORDER_AMT)
    instead of:

    sum((a11.QTY_SOLD * (a11.UNIT_PRICE - a11.DISCOUNT)))
    Labels:

    Comments

    1. hinaFebruary 5, 2021 at 12:19 PM

      Is there any solution now to use aggregate tables with star schema without creating logical tables?

      ReplyDelete

    Post a Comment

    Popular posts from this blog

    Prompt-in-prompt(Nested Prompts) in Microstrategy

    Prompt-in-prompt(Nested Prompts) in  Microstrategy Nested prompts allows you to create one prompt based on the other and other bases on another, nested prompts allows us to prompt the highest level(Like year) to middle level(like Quarter, then to the low level(like Month). Here you can see how to  create a 3-level deep nested prompt that will prompt the user to select a year, then a quarter within that year, then a month within that quarter. Prompt-in-prompt is a feature in which the answer to one prompt is used to define another prompt. This feature is only implemented for element list prompts . The following procedure describes how to achieve this: Create the highest level filter. This is a filter which contains a prompt on an attribute element list. Create a filter on the attribute "Year." Click "prompt on attribute element list" and click "Next" through the rest of the screens to accept the default values. Do not set any additio...
    3 comments
    Read more

    MicroStrategy URL API Parameters

    MicroStrategy URL Structure The following table summarizes the root URL structure used for every request to MicroStrategy Web. Environment Main Application URL Administration URL J2EE http://webserver/MicroStrategy/servlet/mstrWeb http://webserver/MicroStrategy/servlet/mstrWebAdmin .NET http://webserver/MicroStrategy/asp/Main.aspx http://webserver/MicroStrategy/asp/Admin.aspx Every request sent to MicroStrategy Web calls a central controller. Parameters are appended to  Main.aspx  or  mstrWeb  (in a .NET and J2EE environment, respectively) to indicate to the controller how the request should be internally forwarded and handled. The following examples show a URL for accessing a MicroStrategy folder when the user does not have an existing session. The URL contains not only the parameters needed to connect to MicroStrategy Web, but also the parameters needed to log on and create a session. J2EE environment: <a href="http:...
    2 comments
    Read more

    Microstrategy document/dashboard applying selections as filters or slices

    Applying selections as filters or slices In a Microstrategy Document the selections a user makes in a selector can either filter or slice the data in the target: Filtering means that the data for the current selection is calculated only when it is requested by the user. The selections are used to filter the underlying dataset before the metric values are aggregated at the level of the Grid/Graph that is displayed in the document. If the source attribute is not included in the Grid/Graph, the metric values from all the selected elements are aggregated and shown at the level specified in the Grid/Graph. All metric condition selectors (which filter metric values or ranks) and selectors that target other selectors filter data by default. You cannot change them to slicing selectors. Slicing means that the data for each available item in the selector is calculated in advance when the document is first displayed. Selections made while ...
    2 comments
    Read more

    Custom formatting Microstrategy metric format into 1M 2M etc

    Custom formatting metric values of 1,100,000 into a 1.1 million 1.1M type format  In addition to the the pre-defined options for metric formatting, MicroStrategy supports custom formatting. The MicroStrategy Tutorial project is used to explain how users can customize numbers from "1,000,000" to a "1.00" format. Consider a report containing row data values greater than a million, as illustrated below: To format these metric values to use a decimal (i.e., 1.1) instead of showing all the numerals, right-click on the metric (' Revenue ' , for this example) and select  Formatting > Revenue Values , as shown below: Select " Custom " as a category in the Number tab and enter ' 0,,.## ' (without quotation marks), as shown below: Confirm that the format applied properly:
    2 comments
    Read more

    MicroStrategy VLDB properties with Hive

     Recommended VLDB Properties for use of  MicroStrategy 9 with Hive 0.7x The recommended VLDB optimizations for Hive 0.7x are listed below. These values are set by default when the "Hive 0.7x" database object is used (set at  Configuration Managers > Database Instances > Database Instance > Database connection type ) Selected Default VLDB Properties for Hive 0.7x  VLDB Category  VLDB Property Setting  Value   Tables  Fallback Table Type  Permanent Table  Tables  Maximum SQL Passes Before FallBack   0 (no threshold)  Tables  Maximum Tables in FROM Clause Before FallBack  0 (no threshold)  Tables  Drop Temp Table Method  Drop after final pass   Tables  Table Creation Type  Implicit Table  Query Optimizations   Sub Query Type   Use Temporary Table, falling back to IN (SELECT COL) for cor...
    Post a Comment
    Read more

    Email Subscription in Microstrategy

    Create Email Subscription in Microstrategy Creates an email subscription. Do not include any leading or trailing spaces in the ANSWER parameters. This causes a SQL error and prevents the command from executing. Ex: CREATE EMAILSUBSCRIPTION "New Multi Users" FOR OWNER "administrator" SCHEDULE "Books Closed" CONTACTGROUP "TEST"  CONTENT "Electronics Revenue by Region" IN FOLDER "\Public Objects\REPORTS\SUBJECT Areas\Sales and Profitability Analysis" IN PROJECT "MicroStrategy Tutorial" DELIVERYFORMAT HTML  EXPIRATIONDATE NEVER EXPRIED FILENAME "file_name"   SUBJECT  "Test REPORT" MESSAGE "Please Test"; CREATE EMAILSUBSCRIPTION [ subscription_name ] [FOR OWNER login_name ] SCHEDULE schedule_name  (ADDRESS address_name | USER user_name  | CONTACT contact_name [ADDRESS contact_ address_name ] | CONTACTGROUP contact_group_name ) CONTENT ( report_or_document_name IN FOLDER   loc...
    1 comment
    Read more

    Fiscal Week, Fiscal Month, Fiscal Quarter and Fiscal Year calculations in Microstrategy

    Fiscal Week, Fiscal Month, Fiscal Quarter and Fiscal Year calculations in Microstrategy FiscalWeek Returns the numeric position of a week within a fiscal year, for a given  input date. This function is useful in financial reporting when the start of the fiscal year is different than the start of the calendar year. Syntax FiscalWeek< firstWeekDay ,  firstMonth >( Date / Time ) Where: • Date / Time  is the input date or timestamp. • firstWeekDay  (default value is 1) is a parameter that determines which day of the week is considered as the first day of the week. You can type an integer value from 1 to 7, with 1 representing Sunday, 2 representing Monday, and so on until 7 representing Saturday. • firstMonth  (default value is 1) is a parameter that determines which month is considered as the start of the fiscal year. You can type an integer value from 1 to 12, with 1 representing January, 2 representing February, and so on until ...
    1 comment
    Read more

    Configure Connection Mapping in Microstrategy

    Configure Connection Mapping in Microstrategy The following steps demonstrate the second scenario where two different data warehouses are used within the same project: Create two different database connections -                                                                                        One that points to the data warehouse for the European users                                                                 and the other that points to the data warehouse for USA users as shown below: Select Europe as the default database connection for the database Instance as seen below: Go to P...
    Post a Comment
    Read more

    Microstrategy Authentication Using the URL API

    Microstrategy Authentication Using the URL API Users have to be authenticated before accessing functionality in MicroStrategy Web. Using the URL API, there are three ways for MicroStrategy Web to obtain the information needed to authenticate a user. Opening the login page to gather user for credentials    Bypassing the login page by providing credentials in the URL    Bypassing the login page by providing the session state in the URL A detailed explanation of each method for obtaining the authentication information is provided below. Opening the login page to gather user for credentials If the URL attempts to access a MicroStrategy Web page that requires login and no credentials or session state are provided in the URL, the user is redirected to the login page. If login is successful, the user is redirected to the specified page.   The sample URL shown below executes a report without providing authenticating information. Since the Repo...
    Post a Comment
    Read more