Skip to main content

Star schemas and aggregate (or summary) fact tables


Aggregate tables can further improve query performance by reducing the number of rows over which higher-level metrics must be aggregated. 
However, the use of aggregate tables with dimension tables is not a valid physical modeling strategy. Whenever aggregation is performed over fact data, it is a general requirement that tables joined to the fact table must be at the same attribute level or at a higher level. If the auxiliary table is at a lower level, fact rows will be replicated prior to aggregation and this will result in inflated metric values (also known as "multiple counting").

With the above Time dimension table, a fact table at the level of Day functions correctly because there is exactly one row in DIM_TIME for each day. To aggregate the facts to the level of Quarter, it is valid to join the fact table to the dimension table and group by the quarter ID from the dimension table.

Sql
select DT.QUARTER_ID,
   max(DT.QUARTER_DESC) Quarter_Desc
   sum(FT.REVENUE) Revenue
from DAY_FACT_TABLE FT
   join DIM_TIME DT
      on (FT.DAY_ID = DT.DAY_ID)
group by DT.QUARTER_ID

If, however, there is an aggregate fact table already at the level of Quarter, the results will not be correct. This is because the query must join on Quarter ID, but the quarter ID is not a unique key of the dimension table. Because any given quarter of a year contains 90, 91 or 92 days, the dimension table will contain that many rows with the same quarter ID. Thus fact rows will be replicated prior to taking the sum, and the sum will be too high.

Sql
select FT.QUARTER_ID,
   max(DT.QUARTER_DESC) Quarter_Desc
   sum(FT.REVENUE) Revenue
from QTR_FACT_TABLE FT
   join DIM_TIME DT
      on (FT.QUARTER_ID = DT.QUARTER_ID)
group by FT.QUARTER_ID

This is a generally recognized problem with star schemas, and is not strictly a MicroStrategy limitation.

Star schemas will function correctly with MicroStrategy SQL Generation Engine 8.x as long as they obey the general data warehousing principle that fact tables should not be at a higher level than the dimension tables to which they are joined.

If aggregate tables are required, it is necessary to provide higher-level lookup tables with unique rows corresponding to each aggregate table's key. Logical views are a way to do this without adding tables or views to the warehouse. For example, LWV_LU_QUARTER may be defined using the following SQL statement:

Sql
select distinct QUARTER_ID, QUARTER_DESC, YEAR_ID
from DIM_TIME

 

With this logical view, it becomes possible for MicroStrategy SQL Generation Engine 8.x to query the quarter-level fact table as follows. Since the logical view has distinct rows per quarter, multiple counting will not occur in this query.

Sql
select FT.QUARTER_ID,
   max(LQ.QUARTER_DESC) Quarter_Desc
   sum(FT.REVENUE) Revenue
from QTR_FACT_TABLE FT
   join (select distinct QUARTER_ID, QUARTER_DESC, YEAR_ID
         from DIM_TIME) LQ
      on (FT.QUARTER_ID = LQ.QUARTER_ID)
group by FT.QUARTER_ID

For more information on the use of logical views in MicroStrategy SQL Generation Engine 8.1.x and 9.x, consult the MicroStrategy Project Design Guide manual, Appendix B: Logical Tables, "Creating logical tables."

Aggregate tables store pre-summarized totals at a higher level of aggregation than the most granular fact table. They allow reports to be generated from small, rather than large, tables; therefore, performance is enhanced. A successful aggregation strategy seeks to choose aggregate tables that will have the most impact while taking the least amount of space.

Aggregation decisions are driven by the following factors:

  • Usage patterns: Build aggregate tables that are likely to be used the most.
  • Compression ratios: The compression ratio between two tables is defined as the size of the aggregate compared to the size of the smallest table from which the aggregate can be derived.
  • Volatility: Changes in hierarchies over time impact the accuracy of aggregate tables. Sometimes aggregate tables must be rebuilt as a result of changes in dimensions.
A good candidate for aggregation should have at most 10-15 percent of the size of the smallest table from which it is derived.

EXAMPLE:
The MicroStrategy Tutorial project uses aggregate tables by default. A simple metric sum (Revenue) will go to different aggregate tables depending on the attributes on the template.

  1. Create a report with Year on the rows and Revenue on the columns.
  2. Execute the report and view the SQL:

  3. Drill from Year to Item and view the SQL:

The query will go from using ORDER_FACT to ORDER_DETAIL. When Year is on the template, the engine selects the smaller table and the fact is calculated as:

sum(a11.ORDER_AMT)
    instead of:

    sum((a11.QTY_SOLD * (a11.UNIT_PRICE - a11.DISCOUNT)))
    Labels:

    Comments

    1. hinaFebruary 5, 2021 at 12:19 PM

      Is there any solution now to use aggregate tables with star schema without creating logical tables?

      ReplyDelete

    Post a Comment

    Popular posts from this blog

    Case functions Microstrategy

    Ca se functions Microstrategy Case functions return specified data in a SQL query based on the evaluation of user-defined conditions. In general, a user specifies a list of conditions and corresponding return values. Case This function evaluates multiple expressions until a condition is determined to be true, then returns a corresponding value. If all conditions are false, a default value is returned.  Case  can be used for categorizing data based on multiple conditions. This is a single-value function. Syntax Case ( Condition1 ,  ReturnValue1 ,  Condition2 , ReturnValue2 ,...,  DefaultValue ) Example Case(([Total Revenue] < 300000), 0, ([Total Revenue] < 600000), 1, 2) sum(Case (Day@DESC in (“Sat”,”Sun”), Sales, 0) {~+} Sum(Case(Category@DESC In("Books","Electronics"),Revenue,0)){~+} CaseV (case vector) CaseV  evaluates a single metric and returns different values according to the results. It can be used to perfo...
    3 comments
    Read more

    HTML image as a button that exports a grid on a document to Excel in MicroStrategy

    HTML Image as a Button That Exports a Grid on a Document to Excel in MicroStrategy In MicroStrategy Web, it is possible to use an image as a button to export a grid on a document to Excel in MicroStrategy version 10.4 and later. This document provides the steps to accomplish the same.  The goal is to create an image link that is used to export the grid, like shown below: Document in Edit Mode Follow the steps below: 1. Create a Report Services Document 2. Make two grids with different attributes and metrics in each  3. Insert an HTML   container   (and not an image) and insert the script below. Make sure the Project name matches the given project and gridKey matches ( highlighted in bold in the script below). The code below is used for the first grid and button. Main things to watch are the  gridKey  values and the name of the element id which in the below code is  exportLink .   <a target="_blank" id="exportLink" href=""...
    2 comments
    Read more

    mstrio – Python and R wrappers for the MicroStrategy

    mstrio – Python and R wrappers for the MicroStrategy REST APIs Connecting to MicroStrategy  Create a connection to the Intelligence Server using   Connection()   and    connect()  in Python and R, respectively. Required arguments for the   Connection()  function are the URL for the MicroStrategy REST API server, MicroStrategy Intelligence Server username and password, as well as the MicroStrategy project name. By default, the   connect()  function anticipates your MicroStrategy Intelligence Server username and password. LDAP authentication is also supported. Use the optional argument    login_mode=16    in the    connect()  function for LDAP authentication.  Extract data from cubes and reports  To extract data from MicroStrategy cubes and reports, use the   get_cube()  and   get_report()  functions. Use...
    8 comments
    Read more

    Microstrategy Custom number formatting symbols

    Custom number formatting symbols If none of the built-in number formats meet your needs, you can create your own custom format in the Number tab of the Format Cells dialog box. Select  Custom  as the Category and create the format using the number format symbols listed in the table below. Each custom format can have up to four optional sections, one each for: Positive numbers Negative numbers Zeros Text Each section is optional. Separate the sections by semicolons, as shown in the example below: #,###;(#,###);0;"Error: Entry must be numeric" For more examples, see  Custom number formatting examples . To jump to a section of the formatting symbol table, click one of the following: Numeric symbols Character/text symbols Date and time symbols Text color symbols Currency symbols Conditional symbols Numeric symbols For details on how numeric symbols apply to the Big Decimal data type, refer to the  Project Design Guide . ...
    3 comments
    Read more

    Conversion failed when converting the varchar value 'xxxx' Microstartegy

    Error "Conversion failed  Error "Conversion failed when converting the varchar value 'xxxx' to data type int" happens when displaying Picture type attribute form using ApplySimple in expression against SQL Server 2012 in MicroStrategy  The attribute form is in Picture type and defined with the following ApplySimple function with Int type column [ID_BARANG] as the input parameter against SQL Server 2012.  Solutions is to use  Concat("Images/demo/s", [BARANG_ID_INT], ".png") ApplySimple("'images/demo/'&#0&'.png'", [ID_BARANG]) However, when running reports with attribute to show the picture form in Web, error message happens in both Web and Developer. Conversion failed when converting the varchar value 'images/demo/s' to data type int. STEPS TO REPRODUCE: SQL Server 2012 database should be used as the warehouse.  Create an attribute form as type Picture and us custom expressi...
    1 comment
    Read more

    Microstrategy Release Types and Scheduling

    Microstrategy Release Types and Scheduling Organizations that use MicroStrategy have asked for a more frequent and predictable platform release schedule—and we listened! We want you to be able to adopt the latest innovations more regularly while having the ability to budget for your long-term upgrade roadmap. Recently announced by CTO Tim Lang in his newsletter, that’s why we’re adopting an annual cadence for future platform releases. This means that our upcoming platform release (v11.1) hits the streets in Q4 2018, followed by another platform release in Q4 2019. PLATFORM RELEASE Interval:  Annually every twelve (12) months in December Who:  Entire customer base What:  Focus on production level security, stability, and performance defect fixes for all customers. Expectation:  Customer has chosen platform path and wants product stability without new enhancements. Support:  Three (3) years, patches for approved P1 defects, an...
    1 comment
    Read more

    Create an alert-based subscription in MicroStrategy Distribution Services

    Create an alert-based subscription in MicroStrategy Distribution Services on Web Subscription to a report or Report Services document which will be executed when a certain conditional threshold is met based on another executing report. For example, a scheduled report executes which shows the Revenue by day for the past week. If the Revenue on any one day falls below a certain value, a subscription to another report or Report Services document can be triggered and delivered to a recipient. An alert based subscription can only be created directly on a report; however, another report or Report Services document can be delivered when the alert based subscription is triggered. Note: you need a grid report to create an alert and you cannot create if you want to create on a document with text boxes. The following example will walk through the basic steps on how to setup a subscription based on an alert like this: Follow the brief  steps bel...
    3 comments
    Read more

    Level metrics in Microstartegy/MSTR

    Level Metrics Level metrics allows the users to choose the right combination of level target, filtering, and grouping (referred to as elements) to achieve your desired results.  The elements of a metric level are described below: • Target : The target is the attribute level at which the metric calculation groups. For a more detailed description, see  Target: The context of a calculation . • Grouping : Grouping determines how the metric aggregates. For a more detailed description, see  Grouping: How aggregation is performed . • Filtering : Filtering governs how the report filter interacts with the metric calculation. For a more detailed description, see  Filtering: Interaction with report filters . The level is indicated between the curly braces ({ }) in the metric definition shown below: Sum(Revenue) {~, Product} The tilde (~) represents the report level with standard filtering, denoted by the plus sign (+). If you add item as a level, th...
    Post a Comment
    Read more

    Stages in MicroStrategy Report Execution Process

    Stages in MicroStrategy Report Execution Process The report execution is typically a 3 step process process of the below: Query Stage Populate & Evaluate Stage Cross-tab stage Query Stage:       Retrieve data from the warehouse  Populate & Evaluate Stage:       Fill report data for display Cross-tab Stage:      Pivot and display sorting and page-by 
    1 comment
    Read more
    MicroStrategy Developer Preferences options are expanded so big that some options are being cutoff. Show the hidden objects in the  Microstrategy  developer MicroStrategy Developer Preferences options are expanded so big that some options are being cut off. The steps below given in the MSTR article may not work. This can be simple handled by using the steps below:  In the Microstrategy Developer go to Tools -> Preferences (Not my prefernces :) ) Under Developer category -> select Browsing on the browsing tab you see all the options like below: 3. Now using the mouse place the cursor on text box of 10000 which is next to 'Maximum number of monitoring objects displayed per page. 4. Then Hit Tab on Keyboard and hit another Tab on keyboard 5. Then press the space or down arrow on keyboard and click on OK or Enter. That will show the hidden objects in the Microstrategy developer   Normal Version ...
    Post a Comment
    Read more