Previous Next  

  Overview: Execution Packages

Execution Packages allow users to define a set of tables and configure how they should be executed. Take, for example, an organization that wants to refresh all of their data on a nightly basis. They would create an Execution Package that includes the entire project, and schedule it to run daily at 10pm.

Default Package

The default execution package is the package used when running a manual execution from within the project. It is important you do not add steps into the include steps of this package, unless you want them to be run during every manual execution. However, you may modify the setup options in this package to help improve the performance of your manual executions. For example, it may be ideal for you to increase the Max. Threads and set Managed Execution to "Execution Time" on this package.

Include/Exclude Steps

In the Execution Package dialog, project objects are broken down into "Steps". The "All Steps" pane includes all steps currently available in your project. This includes Databases, Tables, Semantic Models, Cubes, other Execution Packages, and Project Perspectives. You can drag objects from the "All Steps" pane into the "Include Steps" pane to include them in the package. Objects dragged into the "Exclude Steps" pane will be ignored or skipped during execution of the package.

Full Load

It may be desired to occasionally reload, or Full Load, incremental tables to fully refresh all historical records in those tables. The Full load objects tab shows all objects with Incremental Load enabled and allows you to drag them into the "Full Load Objects" pane to fully load these objects during the execution of the package.

Setup Options

The Execution Setup Options allow you to configure how you want your package to be run.

Max Threads

This setting allows you to set the maximum number of steps that may run in parallel during a single execution. Depending on your server resources, increasing this setting can significantly reduce overall execution time.

While CPU speed can affect how many threads can be run optimally, this selection is not determined by the number of physical CPU cores on the server. Optimal configuration of this setting can depend on many different factors unique to your server environment. While to few threads can cause slower than necessary execution times, too many threads can cause your entire server to become slow or even unresponsive. It is recommended to test variations in this setting to determine the most optimal configuration for your environment.

Managed Execution

One of the benefits of Discovery Hub's metadata driven model is that the platform can keep track of all object dependencies within a project. Discover Hub uses this metadata to ensure all objects are executed in the necessary order using a method called Managed Execution. This drop-down allows you to choose one of 3 Managed execution types, or disable it entirely.

  • Execution Time prioritizes objects based on their average execution time so the objects with the longest running execution is executed as early as possible, taking object dependency into account. This option will result in the shortest execution time in most cases.
  • Execution Number prioritizes objects based on their position in the project tree from top to bottom, taking dependency into account.
  • Classification prioritizes objects based on 4 table classifications in the following order: Fact Table - Large, Fact Table, Dimension Table - Large, Dimension Table. These classifications can be assigned to each individual table in "Table Settings".
  • Disabled will disable Managed Execution and execute objects based on their position in the object tree, ignoring dependencies.

Log Execution Time & Log Row Counts

Discovery Hub records the time and row counts of each execution for later analysis, however this does add some additional time to the total execution. Un-checking these boxes will disable these steps from the execution package and slightly reduce the overall execution time.

Merge Steps

Checking this box will "Merge" all steps added to the Include Steps pane. Meaning it will treat all sub-steps as a single entity and apply managed execution to those sub-steps accordingly. If this box is un-checked Discovery Hub will complete the steps in the order they appear in the Include Steps pane. For Example, in a production scenario, you may want to ensure all of the denormalization and transformation work is successful before modifying data in the MDW. In this case you would drag the ODX, DSA, and MDW into the Include Steps pane, in that order, and un-check "Merge Steps". This would complete the ODX before starting the DSA, then complete the DSA before starting the MDW.

Prioritization

Prioritizations that are created in the Execution tab will appear here. Prioritizations are a set of objects that you desire to be executed first. This can be useful if you, for instance, only have a small time slot for extracting data from a source, or if you would like to have a certain cube ready for the users as early as possible during execution.


Failure Handling

This group of settings allow you to configure a certain number of retries when executing a package. This can be helpful if you have poor network connectivity and wish to retry after a few errors.

Post Execution

These options allow you to initiate notifications or other Execution packages once the current package is complete.

Notifications

Notifications can be configured in the Execution Tab. They enable you to configure notifications that can be sent to an email address, the windows event log, or both. The notifications can include the package name, project name, and the full execution message.

Run Package

This drop-down allows you to select another execution package to be run after the current one is completed.

Discussion
0 comments