Skip to main content

Star schemas and aggregate (or summary) fact tables


Aggregate tables can further improve query performance by reducing the number of rows over which higher-level metrics must be aggregated. 
However, the use of aggregate tables with dimension tables is not a valid physical modeling strategy. Whenever aggregation is performed over fact data, it is a general requirement that tables joined to the fact table must be at the same attribute level or at a higher level. If the auxiliary table is at a lower level, fact rows will be replicated prior to aggregation and this will result in inflated metric values (also known as "multiple counting").

With the above Time dimension table, a fact table at the level of Day functions correctly because there is exactly one row in DIM_TIME for each day. To aggregate the facts to the level of Quarter, it is valid to join the fact table to the dimension table and group by the quarter ID from the dimension table.

Sql
select DT.QUARTER_ID,
   max(DT.QUARTER_DESC) Quarter_Desc
   sum(FT.REVENUE) Revenue
from DAY_FACT_TABLE FT
   join DIM_TIME DT
      on (FT.DAY_ID = DT.DAY_ID)
group by DT.QUARTER_ID

If, however, there is an aggregate fact table already at the level of Quarter, the results will not be correct. This is because the query must join on Quarter ID, but the quarter ID is not a unique key of the dimension table. Because any given quarter of a year contains 90, 91 or 92 days, the dimension table will contain that many rows with the same quarter ID. Thus fact rows will be replicated prior to taking the sum, and the sum will be too high.

Sql
select FT.QUARTER_ID,
   max(DT.QUARTER_DESC) Quarter_Desc
   sum(FT.REVENUE) Revenue
from QTR_FACT_TABLE FT
   join DIM_TIME DT
      on (FT.QUARTER_ID = DT.QUARTER_ID)
group by FT.QUARTER_ID

This is a generally recognized problem with star schemas, and is not strictly a MicroStrategy limitation.

Star schemas will function correctly with MicroStrategy SQL Generation Engine 8.x as long as they obey the general data warehousing principle that fact tables should not be at a higher level than the dimension tables to which they are joined.

If aggregate tables are required, it is necessary to provide higher-level lookup tables with unique rows corresponding to each aggregate table's key. Logical views are a way to do this without adding tables or views to the warehouse. For example, LWV_LU_QUARTER may be defined using the following SQL statement:

Sql
select distinct QUARTER_ID, QUARTER_DESC, YEAR_ID
from DIM_TIME

 

With this logical view, it becomes possible for MicroStrategy SQL Generation Engine 8.x to query the quarter-level fact table as follows. Since the logical view has distinct rows per quarter, multiple counting will not occur in this query.

Sql
select FT.QUARTER_ID,
   max(LQ.QUARTER_DESC) Quarter_Desc
   sum(FT.REVENUE) Revenue
from QTR_FACT_TABLE FT
   join (select distinct QUARTER_ID, QUARTER_DESC, YEAR_ID
         from DIM_TIME) LQ
      on (FT.QUARTER_ID = LQ.QUARTER_ID)
group by FT.QUARTER_ID

For more information on the use of logical views in MicroStrategy SQL Generation Engine 8.1.x and 9.x, consult the MicroStrategy Project Design Guide manual, Appendix B: Logical Tables, "Creating logical tables."

Aggregate tables store pre-summarized totals at a higher level of aggregation than the most granular fact table. They allow reports to be generated from small, rather than large, tables; therefore, performance is enhanced. A successful aggregation strategy seeks to choose aggregate tables that will have the most impact while taking the least amount of space.

Aggregation decisions are driven by the following factors:

  • Usage patterns: Build aggregate tables that are likely to be used the most.
  • Compression ratios: The compression ratio between two tables is defined as the size of the aggregate compared to the size of the smallest table from which the aggregate can be derived.
  • Volatility: Changes in hierarchies over time impact the accuracy of aggregate tables. Sometimes aggregate tables must be rebuilt as a result of changes in dimensions.
A good candidate for aggregation should have at most 10-15 percent of the size of the smallest table from which it is derived.

EXAMPLE:
The MicroStrategy Tutorial project uses aggregate tables by default. A simple metric sum (Revenue) will go to different aggregate tables depending on the attributes on the template.

  1. Create a report with Year on the rows and Revenue on the columns.
  2. Execute the report and view the SQL:

  3. Drill from Year to Item and view the SQL:

The query will go from using ORDER_FACT to ORDER_DETAIL. When Year is on the template, the engine selects the smaller table and the fact is calculated as:

sum(a11.ORDER_AMT)
    instead of:

    sum((a11.QTY_SOLD * (a11.UNIT_PRICE - a11.DISCOUNT)))
    Labels:

    Comments

    1. hinaFebruary 5, 2021 at 12:19 PM

      Is there any solution now to use aggregate tables with star schema without creating logical tables?

      ReplyDelete

    Post a Comment

    Popular posts from this blog

    Create an alert-based subscription in MicroStrategy Distribution Services

    Create an alert-based subscription in MicroStrategy Distribution Services on Web Subscription to a report or Report Services document which will be executed when a certain conditional threshold is met based on another executing report. For example, a scheduled report executes which shows the Revenue by day for the past week. If the Revenue on any one day falls below a certain value, a subscription to another report or Report Services document can be triggered and delivered to a recipient. An alert based subscription can only be created directly on a report; however, another report or Report Services document can be delivered when the alert based subscription is triggered. Note: you need a grid report to create an alert and you cannot create if you want to create on a document with text boxes. The following example will walk through the basic steps on how to setup a subscription based on an alert like this: Follow the brief  steps bel...
    2 comments
    Read more

    MicroStrategy URL API Parameters

    MicroStrategy URL Structure The following table summarizes the root URL structure used for every request to MicroStrategy Web. Environment Main Application URL Administration URL J2EE http://webserver/MicroStrategy/servlet/mstrWeb http://webserver/MicroStrategy/servlet/mstrWebAdmin .NET http://webserver/MicroStrategy/asp/Main.aspx http://webserver/MicroStrategy/asp/Admin.aspx Every request sent to MicroStrategy Web calls a central controller. Parameters are appended to  Main.aspx  or  mstrWeb  (in a .NET and J2EE environment, respectively) to indicate to the controller how the request should be internally forwarded and handled. The following examples show a URL for accessing a MicroStrategy folder when the user does not have an existing session. The URL contains not only the parameters needed to connect to MicroStrategy Web, but also the parameters needed to log on and create a session. J2EE environment: <a href="http:...
    2 comments
    Read more

    Save Personal View of Microstrategy Report/Dashboard

    Save Personal View of Microstrategy Report/Dashboard Starting in MicroStrategy Web 9.3.0, it is possible to capture a view of the report / document that user is viewing at a particular moment. This feature is available for both Report and Document. Create Personal View option is located under Home Menu bar (see image below). Saved view is available under My Subscription page under Personal View Subscriptions (see second image below) Create Personal View option is located under Home Menu bar Saved view is available under My Subscription page under Personal View Subscriptions Saving a personal view of a report, document, or quick dashboard A personal view allows you to save your own custom view of a report, document, or quick dashboard. The personal view includes every change that you made to the report, document, or quick dashboard since you opened it, including: Creating metrics Filtering Pivoting Sorting Prompt answers for a r...
    Post a Comment
    Read more

    Fact tables levels tables in Microstrategy explained

    Fact tables levels in Microstrategy: Fact tables are used to store fact data. Fact tables should contain attribute Id's and fact values which are measurable. All the descriptive information about the fact tables should stored in Dimension tables either in Star Schema fashion or Snow Flake Schema fashion which is best suited to your reporting solution. Since attributes provide context for fact values, both fact columns and attribute ID columns are included in fact tables. Facts help to link indirectly related attributes using these attribute ID columns. The attribute ID columns included in a fact table represent the level at which the facts in that table are stored. So the level of a fact table in the Fact_Item_Day_Customer can be the attribute Id's which is at Day, Item & Customer Id level. For example, fact tables containing sales and inventory data look like the tables shown in the following diagram: Base fact columns ver...
    2 comments
    Read more

    Case functions Microstrategy

    Ca se functions Microstrategy Case functions return specified data in a SQL query based on the evaluation of user-defined conditions. In general, a user specifies a list of conditions and corresponding return values. Case This function evaluates multiple expressions until a condition is determined to be true, then returns a corresponding value. If all conditions are false, a default value is returned.  Case  can be used for categorizing data based on multiple conditions. This is a single-value function. Syntax Case ( Condition1 ,  ReturnValue1 ,  Condition2 , ReturnValue2 ,...,  DefaultValue ) Example Case(([Total Revenue] < 300000), 0, ([Total Revenue] < 600000), 1, 2) sum(Case (Day@DESC in (“Sat”,”Sun”), Sales, 0) {~+} Sum(Case(Category@DESC In("Books","Electronics"),Revenue,0)){~+} CaseV (case vector) CaseV  evaluates a single metric and returns different values according to the results. It can be used to perfo...
    3 comments
    Read more

    OLAP features in Microstrategy

    OLAP features in Microstrategy MSTR  OLAP Services uses Intelligent Cube Technology—an in-memory version of report data that can 1 About MicroStrategy OLAP Services  can be manipulated by the MicroStrategy Analytical Engine. MicroStrategy Desktop, Web, and Office users can slice and dice data in reports within Intelligent Cubes without having to re-execute SQL against the data warehouse.  Many of the standard OLAP features that MicroStrategy provides out of the box, such as: Page-by Pivoting Sorting Subtotals Banding Aliasing Outline mode Thresholds etc.. With an OLAP Services license, user can perform additional OLAP analysis, using the following features:  Displaying data on the fly: dynamic aggregation, page  Creating metrics on-the-fly: derived metrics, Defining attribute elements on-the-fly: derived elements,  Filtering data on the fly: view filters and metric filters,  Importing data as an Intelligent Cube
    Post a Comment
    Read more

    Clean uninstall MicroStrategy Analytics Enterprise using the Uninstallation Cleanup Utility

    How to  clean uninstall MicroStrategy Analytics Enterprise using the Uninstallation Cleanup Utility CONSIDERATIONS/GUIDELINES : This Uninstallation Cleanup Utility is designed to remove the obsolete or leftover services, files and registries after uninstalling MicroStrategy products.  The goal of this cleanup utility is to remove the leftover files and registries after uninstallation to make the machine "cleaner" for a second time installation (it can solve some known downgrade installation issues).  The basic assumption for removing files is that the user installed MicroStrategy  in a path that contains the string "MicroStrategy" . If not, the files and certain registry keys will not be removed. Using the cleanup utility incorrectly can cause system-wide problems that may require re-installation of the Operating System.  This utility should always be tested in a test environment before being run in a production environment.   Lin...
    1 comment
    Read more

    Multi sourcing in freeform SQL report

    Multi sourcing in freeform SQL report  It is not possible to multisource a Free Form SQL report.If you notice when you create FF SQL report one of the first things is to select the specific data source.   This kind of makes sense because it would be difficult to write SQL that would process on two different DBMS. Options to overcome this issue: (1) Create FF SQL report against one Datasource and configure Data mart to another.   Then all data is in one system. (2) Create a DB_LINK between the two sources.    The SQL can access tables from both.   If you want more security on this method you can create views on specific linked tables and limit access for MicroStrategy DB Connection user to the views.Sample Syntax for SQL Server Link to Oracle.First set up ODBC to Oracle instance.   And run the following script with your settings.
    Post a Comment
    Read more

    Prompt-in-prompt(Nested Prompts) in Microstrategy

    Prompt-in-prompt(Nested Prompts) in  Microstrategy Nested prompts allows you to create one prompt based on the other and other bases on another, nested prompts allows us to prompt the highest level(Like year) to middle level(like Quarter, then to the low level(like Month). Here you can see how to  create a 3-level deep nested prompt that will prompt the user to select a year, then a quarter within that year, then a month within that quarter. Prompt-in-prompt is a feature in which the answer to one prompt is used to define another prompt. This feature is only implemented for element list prompts . The following procedure describes how to achieve this: Create the highest level filter. This is a filter which contains a prompt on an attribute element list. Create a filter on the attribute "Year." Click "prompt on attribute element list" and click "Next" through the rest of the screens to accept the default values. Do not set any additio...
    3 comments
    Read more

    Execute Integrity manager test from Command line

    Execute Integrity manager test from Command line  MSTR Integrity Manager allows the user to execute a test without having to load the GUI, or to schedule a test to run later at specific times or dates. Go over using Windows AT command at: https://support.microsoft.com/en-us/help/313565/how-to-use-the-at-command-to-schedule-tasks https://www.lifewire.com/at-command-2618090 Here are the prerequisites to execute a test from the command line: Create a test and saved using the Integrity Manager graphical interface. Make sure that the users has the ' Use Integrity Manager ' privilege for that project(provided by the administrator) and the ' Execute ' permission for the reports to be tested. Keep in mind that MicroStrategy Integrity Manager can only test three-tier projects, i.e., projects which are connected to a MicroStrategy i- Server. Projects in Direct Connection (two-tier) mode cannot be tested with this tool...
    1 comment
    Read more