Skip to main content

Star schemas and aggregate (or summary) fact tables


Aggregate tables can further improve query performance by reducing the number of rows over which higher-level metrics must be aggregated. 
However, the use of aggregate tables with dimension tables is not a valid physical modeling strategy. Whenever aggregation is performed over fact data, it is a general requirement that tables joined to the fact table must be at the same attribute level or at a higher level. If the auxiliary table is at a lower level, fact rows will be replicated prior to aggregation and this will result in inflated metric values (also known as "multiple counting").

With the above Time dimension table, a fact table at the level of Day functions correctly because there is exactly one row in DIM_TIME for each day. To aggregate the facts to the level of Quarter, it is valid to join the fact table to the dimension table and group by the quarter ID from the dimension table.

Sql
select DT.QUARTER_ID,
   max(DT.QUARTER_DESC) Quarter_Desc
   sum(FT.REVENUE) Revenue
from DAY_FACT_TABLE FT
   join DIM_TIME DT
      on (FT.DAY_ID = DT.DAY_ID)
group by DT.QUARTER_ID

If, however, there is an aggregate fact table already at the level of Quarter, the results will not be correct. This is because the query must join on Quarter ID, but the quarter ID is not a unique key of the dimension table. Because any given quarter of a year contains 90, 91 or 92 days, the dimension table will contain that many rows with the same quarter ID. Thus fact rows will be replicated prior to taking the sum, and the sum will be too high.

Sql
select FT.QUARTER_ID,
   max(DT.QUARTER_DESC) Quarter_Desc
   sum(FT.REVENUE) Revenue
from QTR_FACT_TABLE FT
   join DIM_TIME DT
      on (FT.QUARTER_ID = DT.QUARTER_ID)
group by FT.QUARTER_ID

This is a generally recognized problem with star schemas, and is not strictly a MicroStrategy limitation.

Star schemas will function correctly with MicroStrategy SQL Generation Engine 8.x as long as they obey the general data warehousing principle that fact tables should not be at a higher level than the dimension tables to which they are joined.

If aggregate tables are required, it is necessary to provide higher-level lookup tables with unique rows corresponding to each aggregate table's key. Logical views are a way to do this without adding tables or views to the warehouse. For example, LWV_LU_QUARTER may be defined using the following SQL statement:

Sql
select distinct QUARTER_ID, QUARTER_DESC, YEAR_ID
from DIM_TIME

 

With this logical view, it becomes possible for MicroStrategy SQL Generation Engine 8.x to query the quarter-level fact table as follows. Since the logical view has distinct rows per quarter, multiple counting will not occur in this query.

Sql
select FT.QUARTER_ID,
   max(LQ.QUARTER_DESC) Quarter_Desc
   sum(FT.REVENUE) Revenue
from QTR_FACT_TABLE FT
   join (select distinct QUARTER_ID, QUARTER_DESC, YEAR_ID
         from DIM_TIME) LQ
      on (FT.QUARTER_ID = LQ.QUARTER_ID)
group by FT.QUARTER_ID

For more information on the use of logical views in MicroStrategy SQL Generation Engine 8.1.x and 9.x, consult the MicroStrategy Project Design Guide manual, Appendix B: Logical Tables, "Creating logical tables."

Aggregate tables store pre-summarized totals at a higher level of aggregation than the most granular fact table. They allow reports to be generated from small, rather than large, tables; therefore, performance is enhanced. A successful aggregation strategy seeks to choose aggregate tables that will have the most impact while taking the least amount of space.

Aggregation decisions are driven by the following factors:

  • Usage patterns: Build aggregate tables that are likely to be used the most.
  • Compression ratios: The compression ratio between two tables is defined as the size of the aggregate compared to the size of the smallest table from which the aggregate can be derived.
  • Volatility: Changes in hierarchies over time impact the accuracy of aggregate tables. Sometimes aggregate tables must be rebuilt as a result of changes in dimensions.
A good candidate for aggregation should have at most 10-15 percent of the size of the smallest table from which it is derived.

EXAMPLE:
The MicroStrategy Tutorial project uses aggregate tables by default. A simple metric sum (Revenue) will go to different aggregate tables depending on the attributes on the template.

  1. Create a report with Year on the rows and Revenue on the columns.
  2. Execute the report and view the SQL:

  3. Drill from Year to Item and view the SQL:

The query will go from using ORDER_FACT to ORDER_DETAIL. When Year is on the template, the engine selects the smaller table and the fact is calculated as:

sum(a11.ORDER_AMT)
    instead of:

    sum((a11.QTY_SOLD * (a11.UNIT_PRICE - a11.DISCOUNT)))

    Labels

    Labels:

    Comments

    1. hinaFebruary 5, 2021 at 12:19 PM

      Is there any solution now to use aggregate tables with star schema without creating logical tables?

      ReplyDelete

    Post a Comment

    Popular posts from this blog

    MicroStrategy URL API Parameters

    MicroStrategy URL Structure The following table summarizes the root URL structure used for every request to MicroStrategy Web. Environment Main Application URL Administration URL J2EE http://webserver/MicroStrategy/servlet/mstrWeb http://webserver/MicroStrategy/servlet/mstrWebAdmin .NET http://webserver/MicroStrategy/asp/Main.aspx http://webserver/MicroStrategy/asp/Admin.aspx Every request sent to MicroStrategy Web calls a central controller. Parameters are appended to  Main.aspx  or  mstrWeb  (in a .NET and J2EE environment, respectively) to indicate to the controller how the request should be internally forwarded and handled. The following examples show a URL for accessing a MicroStrategy folder when the user does not have an existing session. The URL contains not only the parameters needed to connect to MicroStrategy Web, but also the parameters needed to log on and create a session. J2EE environment: <a href="http:...
    2 comments
    Read more

    Case functions Microstrategy

    Ca se functions Microstrategy Case functions return specified data in a SQL query based on the evaluation of user-defined conditions. In general, a user specifies a list of conditions and corresponding return values. Case This function evaluates multiple expressions until a condition is determined to be true, then returns a corresponding value. If all conditions are false, a default value is returned.  Case  can be used for categorizing data based on multiple conditions. This is a single-value function. Syntax Case ( Condition1 ,  ReturnValue1 ,  Condition2 , ReturnValue2 ,...,  DefaultValue ) Example Case(([Total Revenue] < 300000), 0, ([Total Revenue] < 600000), 1, 2) sum(Case (Day@DESC in (“Sat”,”Sun”), Sales, 0) {~+} Sum(Case(Category@DESC In("Books","Electronics"),Revenue,0)){~+} CaseV (case vector) CaseV  evaluates a single metric and returns different values according to the results. It can be used to perfo...
    3 comments
    Read more

    User request is completed. (Ran out of memory)

    Unable to Run/Edit particular MicroStrategy reports ue to the following error: User request is completed. (Ran out of memory) User request is completed. (Ran out of memory) The above issue appeared in MSTR Web Universal version 10.5 We tried the below options without any luck: 1. i-server restart 2. Web server restart 3. clear document cache/dataset cache 4. Web server cache clear as below: The correct option is to increase the contract memory settings: Using the Memory Contract Manager The  MCM settings are in the Intelligence Server Configuration Editor, in the  Governing Rules: Default: Memory Settings  category. The  Enable single memory allocation governing  option lets you specify how much memory can be reserved for a single Intelligence Server operation at a time. When this option is enabled, each memory request is compared to the  Maximum single allocation size (MBytes)  setting. If the request ...
    1 comment
    Read more

    Microstrategy Dashboard performance improvements steps

    Microstrategy  Dashboard performance improvements steps: Many times, causes of poor performance can be simplified to specific components. To troubleshoot performance issues, users must identify these components, then make the appropriate modifications to the environment and/or to the MicroStrategy dashboard to reduce bottlenecks. Dashboard execution stages can be represented below: MicroStrategy Intelligence Server When an end user makes a  Document Execution Request  through any client (a web browser via MicroStrategy Web, the MicroStrategy Desktop/Developer client, the MicroStrategy Mobile app, or the MicroStrategy Office client), the request is sent to the MicroStrategy Intelligence Server, which processes the request and prepares the response. The MicroStrategy Intelligence Server will execute all children datasets on the dashboard by either generating SQL and running this against the data warehouse, or by fetching data from a cache. The Inte...
    1 comment
    Read more

    Create a transaction services photo uploader

    Create a transaction services photo uploader   1.  Create a new table "photo_upload" in Tutorial warehouse database (the default location: C:\Program Files\MicroStrategy\Tutorial Reporting\TUTORIAL_DATA_7200.mdb), as shown below:    2. The 'photo_upload' table has to be pre-populated with *exactly* 10 rows of data, the values for the 'ID' column should be 1-10 and the values for the 'uploaded' column should all be 0 3.  In MicroStrategy Desktop, create a freeform report "R1" based on the new table "photo_upload" in Tutorial data created at step 1, as shown below:   SELECT Location, Description, ID, uploaded, numbers FROM PHOTO_UPLOAD 4.  Create another table for transaction insert SQL. Make sure to create an 'autonumber' type ID as primary key for this table, or auto_increment ID for different DBs.                     5. Create...
    Post a Comment
    Read more

    Multi-Select Drop Down Selector with Apply Button

    Multi-Select Drop Down Selector with  OK Button  When creating a Report Services Document, you may want to create a drop down selector to save space, but also would like the drop down to be a multi-selector with OK/Apply button. Below are instructions to achieve this in a Report Services Document.  Steps to Create: 1. Right click on your drop down selector and choose Properties and Formatting 2. Navigate to the Layout Tab 3. In the layout tab, click "Allow multiple selections". It will  not  indicate that the check box is enabled, but this step is mandatory.  3. Next, navigate to the Theme tab. Choose the  L ight Theme  from the drop down menu and click the Apply button. 4) Next, change the Theme back to  None  and click Apply. 4. When you return to the Layout tab, you will see the checkbox for  Allow multiple selections  is now checked.  
    Post a Comment
    Read more

    Best Practices for using images in Microstrategy reports

    Using images in Microstrategy reports On the I-Server we copy the images to the following folders: - Program Files (x86)\MicroStrategy\Developer\Images - Program Files (x86)\MicroStrategy\IntelligenceServer\Images - Program Files (x86)\Common Files\MicroStrategy\images To reference an image using relative path, the following locations are required to store the image in order to display in multiple MicroStrategy products.    Relative path sample: images/ logo.jpg To display images on: MicroStrategy Desktop, Web, Mobile, Distribution Services: * Additional Adobe Flash security rule applies when displaying images on Flash Dashboard For image to display on MicroStrategy Web Services:   Note: It is recommended that HTTP path should be used for reference image. User should consider using HTTP path to reference image and store image in an client accessible server for easier maintain and upgrading purpose ...
    Post a Comment
    Read more

    Search for Freeform SQL Reports in MicroStrategy Developer

    Search for Freeform SQL Reports in MicroStrategy Developer Steps below will guide to find out all the Freeform SQL Reports in a project in MicroStrategy Developer: Log in to the particular project from Developer and click the search button or use Control +F Go to Tools > Options > Enable "Show ' Contains ' and ' Contained by ' tabs" as shown below: 3.  Click OK > Go to "Contains" tab and select Logical Table for "Contains any object of this type" as shown below:  4.  Then go to Object Types tab and select Report as the object as shown below: 5. Click Find Now and all Freeform SQL Reports will be displayed.
    Post a Comment
    Read more

    Configure a report for use with Bulk Export in MicroStrategy

    Configure a report for use with Bulk Export in MicroStrategy The Bulk Export feature enables a large report to be saved as a delimited text file. Using this feature, it is possible to retrieve result sets from a large dataset without having to load the entire dataset into memory. PS:  Once a report is setup for bulk export it cannot be used as a regular report. So if the report needs to be run as a normal report and as a bulk export report, the first step is to make a copy of the report for use with bulk export. Configure Bulk Export Bulk Export options are only available in MicroStrategy Developer. Open a 3-tier connection using MicroStrategy Developer and edit the desired report. Go to 'Data' on the top menu bar. Select 'Configure Bulk Export': Specify any additional desired configuration options. General Settings Bulk export database instance : This is the database instance to use to store the bulk export results. Temporary tables w...
    Post a Comment
    Read more

    Microstrategy Custom number formatting symbols

    Custom number formatting symbols If none of the built-in number formats meet your needs, you can create your own custom format in the Number tab of the Format Cells dialog box. Select  Custom  as the Category and create the format using the number format symbols listed in the table below. Each custom format can have up to four optional sections, one each for: Positive numbers Negative numbers Zeros Text Each section is optional. Separate the sections by semicolons, as shown in the example below: #,###;(#,###);0;"Error: Entry must be numeric" For more examples, see  Custom number formatting examples . To jump to a section of the formatting symbol table, click one of the following: Numeric symbols Character/text symbols Date and time symbols Text color symbols Currency symbols Conditional symbols Numeric symbols For details on how numeric symbols apply to the Big Decimal data type, refer to the  Project Design Guide . ...
    4 comments
    Read more