Changelog#

0.9.16#

Breaking Changes

• Removed DagsterKubernetesPodOperator in dagster-airflow.
• Removed the execute_plan mutation from dagster-graphql.
• ModeDefinition, PartitionSetDefinition, PresetDefinition, @repository, @pipeline, and ScheduleDefinition names must pass the regular expression r"^[A-Za-z0-9_]+$" and not be python keywords or disallowed names. See DISALLOWED_NAMES in dagster.core.definitions.utils for exhaustive list of illegal names.
• dagster-slack is now upgraded to use slackclient 2.x - this means that this resource will only support Python 3.6 and above.
• [K8s] Added a health check to the helm chart for user deployments, which relies on a new dagster api grpc-health-check cli command present in Dagster 0.9.16 and later.

New

• Add helm chart configurations to allow users to configure a K8sRunLauncher, in place of the CeleryK8sRunLauncher.
• “Copy URL” button to preserve filter state on Run page in dagit

Community Contributions

• Dagster CLI options can now be passed in via environment variables (Thanks @xinbinhuang!)
• New --limit flag on the dagster run list command (Thanks @haydarai!)

Bugfixes

• Addressed performance issues loading the /assets table in dagit. Requires a data migration to create a secondary index by running dagster instance reindex.
• Dagit bugfixes and improvements

0.9.15#

Breaking Changes

• CeleryDockerExecutor no longer requires a repo_location_name config field.
• executeRunInProcess was removed from dagster-graphql.

New

• Dagit: Warn on tab removal in playground
• Display versions CLI: Added a new CLI that displays version information for a memoized run. Called via dagster pipeline list_versions.
• CeleryDockerExecutor accepts a network field to configure the network settings for the Docker container it connects to for execution.
• Dagit will now set a statement timeout on supported instance DBs. Defaults to 5s and can be controlled with the --db-statement-timeout flag

Community Contributions

• dagster grpc requirements are now more friendly for users (thanks @jmo-qap!)
• dagster.utils now has is_str (thanks @monicayao!)
• dagster-pandas can now load dataframes from pickle (thanks @mrdrprofuroboros!)
• dagster-ge validation solid factory now accepts name (thanks @haydarai!)

Bugfixes

• Dagit bugfixes and improvements
• Fixed an issue where dagster could fail to load large pipelines.
• Fixed a bug where experimental arg warning would be thrown even when not using versioned dagster type loaders.
• Fixed a bug where CeleryDockerExecutor was failing to execute pipelines unless they used a legacy workspace config.
• Fixed a bug where pipeline runs using IntMetadataEntryData could not be visualized in dagit.

Experimental

• Improve the output structure of dagster-dbt solids.
• Version-based memoization over outputs stored in the intermediate store now works

Documentation

• Fix a code snippet rendering issue in Overview: Assets & Materializations
• Fixed all python code snippets alignment across docs examples

0.9.14#

New

• Steps down stream of a failed step no longer report skip events and instead simply do not execute.
• dagit-debug can load multiple debug files.
• dagit now has a Debug Console Logging feature flag accessible at /flags .
• Telemetry metrics are now taken when scheduled jobs are executed.
• With memoized reexecution, we now only copy outputs that current plan won't generate
• Document titles throughout dagit

Community Contributions

• [dagster-ge] solid factory can now handle arbitrary types (thanks @sd2k!)
• [dagster-dask] utility options are now available in loader/materializer for Dask DataFrame (thanks @kinghuang!)

Bugfixes

• Fixed an issue where run termination would sometimes be ignored or leave the execution process hanging
• [dagster-k8s] fixed issue that would cause timeouts on clusters with many jobs
• Fixed an issue where reconstructable was unusable in an interactive environment, even when the pipeline is defined in a different module.
• Bugfixes and UX improvements in dagit

Experimental

• AssetMaterializations now have an optional “partition” attribute

0.9.13#

Bugfixes

• Fixes an issue using build_reconstructable_pipeline.
• Improved loading times for the asset catalog in Dagit.

Documentations

• Improved error messages when invoking dagit from the CLI with bad arguments.

0.9.12#

Breaking Changes

• Dagster now warns when a solid, pipeline, or other definition is created with an invalid name (for example, a Python keyword). This warning will become an error in the 0.9.13 release.

Community Contributions

• Added an int type to EventMetadataEntry (Thanks @ChocoletMousse!)
• Added a build_composite_solid_definition method to Lakehouse (Thanks @sd2k!)
• Improved broken link detection in Dagster docs (Thanks @keyz!)

New

• Improvements to log filtering on Run view in Dagit
• Improvements to instance level scheduler page
• Log engine events when pipeline termination is initiated

Bugfixes

• Syntax errors in user code now display the file and line number with the error in Dagit
• Dask executor no longer fails when using intermediate_storage
• In the Celery K8s executor, we now mark the step as failed when the step job fails
• Changed the DagsterInvalidAssetKey error so that it no longer fails upon being thrown

Documentation

• Added API docs for dagster-dbt experimental library
• Fixed some cosmetic issues with docs.dagster.io
• Added code snippets from Solids examples to test path, and fixed some inconsistencies regarding parameter ordering
• Changed to using markers instead of exact line numbers to mark out code snippets