Changelog#

1.8.3 (core) / 0.24.3 (libraries)#

New#

• When different assets within a code location have different PartitionsDefinitions, there will no longer be an implicit asset job __ASSET_JOB_... for each PartitionsDefinition; there will just be one with all the assets. This reduces the time it takes to load code locations with assets with many different PartitionsDefinitions.

1.8.2 (core) / 0.24.2 (libraries)#

New#

• [ui] Improved performance of the Automation history view for partitioned assets
• [ui] You can now delete dynamic partitions for an asset from the ui
• [dagster-sdf] Added support for quoted table identifiers (Thanks, @akbog!)
• [dagster-openai] Add additional configuration options for the OpenAIResource (Thanks, @chasleslr!)
• [dagster-fivetran] Fivetran assets now have relation identifier metadata.

Bugfixes#

• [ui] Fixed a collection of broken links pointing to renamed Declarative Automation pages.
• [dagster-dbt] Fixed issue preventing usage of MultiPartitionMapping with @dbt_assets (Thanks, @arookieds!)
• [dagster-azure] Fixed issue that would cause an error when configuring an AzureBlobComputeLogManager without a secret_key (Thanks, @ion-elgreco and @HynekBlaha!)

Documentation#

• Added API docs for AutomationCondition and associated static constructors.
• [dagster-deltalake] Corrected some typos in the integration reference (Thanks, @dargmuesli!)
• [dagster-aws] Added API docs for the new PipesCloudWatchMessageReader

0.8.5#

Breaking Changes

• Python 3.5 is no longer under test.
• Engine and ExecutorConfig have been deleted in favor of Executor. Instead of the @executor decorator decorating a function that returns an ExecutorConfig it should now decorate a function that returns an Executor.

New

• The python built-in dict can be used as an alias for Permissive() within a config schema declaration.
• Use StringSource in the S3ComputeLogManager configuration schema to support using environment variables in the configuration (Thanks @mrdrprofuroboros!)
• Improve Backfill CLI help text
• Add options to spark_df_output_schema (Thanks @DavidKatz-il!)
• Helm: Added support for overriding the PostgreSQL image/version used in the init container checks.
• Update celery k8s helm chart to include liveness checks for celery workers and flower
• Support step level retries to celery k8s executor

Bugfixes

• Improve error message shown when a RepositoryDefinition returns objects that are not one of the allowed definition types (Thanks @sd2k!)
• Show error message when $DAGSTER_HOME environment variable is not an absolute path (Thanks @AndersonReyes!)
• Update default value for staging_prefix in the DatabricksPySparkStepLauncher configuration to be an absolute path (Thanks @sd2k!)
• Improve error message shown when Databricks logs can't be retrieved (Thanks @sd2k!)
• Fix errors in documentation fo input_hydration_config (Thanks @joeyfreund!)

0.8.4#

Bugfix

• Reverted changed in 0.8.3 that caused error during run launch in certain circumstances
• Updated partition graphs on schedule page to select most recent run
• Forced reload of partitions for partition sets to ensure not serving stale data

New

• Added reload button to dagit to reload current repository
• Added option to wipe a single asset key by using dagster asset wipe <asset_key>
• Simplified schedule page, removing ticks table, adding tags for last tick attempt
• Better debugging tools for launch errors

0.8.3#

Breaking Changes

• Previously, the gcs_resource returned a GCSResource wrapper which had a single client property that returned a google.cloud.storage.client.Client. Now, the gcs_resource returns the client directly.

  To update solids that use the gcp_resource, change:

  context.resources.gcs.client
  

  To:

  context.resources.gcs
  

New

• Introduced a new Python API reexecute_pipeline to reexecute an existing pipeline run.
• Performance improvements in Pipeline Overview and other pages.
• Long metadata entries in the asset details view are now scrollable.
• Added a project field to the gcs_resource in dagster_gcp.
• Added new CLI command dagster asset wipe to remove all existing asset keys.

Bugfix

• Several Dagit bugfixes and performance improvements
• Fixes pipeline execution issue with custom run launchers that call executeRunInProcess.
• Updates dagster schedule up output to be repository location scoped

0.8.2#

Bugfix

• Fixes issues with dagster instance migrate.
• Fixes bug in launch_scheduled_execution that would mask configuration errors.
• Fixes bug in dagit where schedule related errors were not shown.
• Fixes JSON-serialization error in dagster-k8s when specifying per-step resources.

New

• Makes label optional parameter for materializations with asset_key specified.
• Changes Assets page to have a typeahead selector and hierarchical views based on asset_key path.
• dagster-ssh
  • adds SFTP get and put functions to SSHResource, replacing sftp_solid.

Docs

• Various docs corrections

0.8.1#

Bugfix

• Fixed a file descriptor leak that caused OSError: [Errno 24] Too many open files when enough temporary files were created.
• Fixed an issue where an empty config in the Playground would unexpectedly be marked as invalid YAML.
• Removed "config" deprecation warnings for dask and celery executors.

New

• Improved performance of the Assets page.