snowflake_stage recreates every apply regardless of an actual change #2679

bmhah · 2024-04-05T20:33:52Z

Terraform CLI and Provider Versions

v1.5.0
0.85.0

Terraform Configuration

resource "snowflake_stage" "LANDING_8X8_STAGE_INBOUND_CALLS_S3" {
  name                = "LANDING_8X8_STAGE_INBOUND_CALLS"
  url                 = var.s3_uri_8x8_inbound_calls
  database            = "AWS_LANDING_INGEST_DB_${var.SF_ENVIRONMENT}"
  schema              = "LANDING_8X8_SCHEMA"
  comment             = "Landing stage for inbound calls s3 bucket"
  file_format         = "FORMAT_NAME = AWS_LANDING_INGEST_DB_${var.SF_ENVIRONMENT}.LANDING_8X8_SCHEMA.LANDING_FF_CSV"
  copy_options        = "ON_ERROR = 'CONTINUE'"
  storage_integration = var.aws_storage_integration
}

Expected Behavior

Stage doesn't destroy and recreate

Actual Behavior

Stage recreates, this causes snowflake pipes to break having a status STOPPED_STAGE_DROPPED

Steps to Reproduce

terraform apply

How much impact is this issue causing?

High

Logs

No response

Additional Information

No response

The text was updated successfully, but these errors were encountered:

sfc-gh-asawicki · 2024-04-08T08:19:05Z

Hey @bmhah. Thanks for reaching out to us.

Please share the terraform run logs with TF_LOG=DEBUG enabled.

sfc-gh-bhill · 2024-04-15T20:31:24Z

Here is the file that the customer provided me. plan.txt.gz

bmhah · 2024-04-17T19:37:47Z

Any updates here?

sfc-gh-asawicki · 2024-04-17T20:24:01Z

Hey @bmhah. I will confirm in the morning, but it looks like an easy fix, so we may even be able to include it in tomorrow's release. I will keep you posted.

References: #2679

- Fix number of allowed values in tags - Suppress diff for stages `on_error`

🤖 I have created a release *beep* *boop* --- ## [0.89.0](v0.88.0...v0.89.0) (2024-04-18) ### 🎉 **What's new:** * Update target objects for privilege-granting resources ([#2688](#2688)) ([74e2b6b](74e2b6b)) ### 🔧 **Misc** * Handle generic check destroy in acceptance tests ([#2716](#2716)) ([63a5324](63a5324)) * Initialize the SDK client in fewer places ([#2710](#2710)) ([382bfc1](382bfc1)) ### 🐛 **Bug fixes:** * Adds case statement for ObjectTypeUser in ShowObjectParameter. ([#2675](#2675)) ([23a3341](23a3341)) * diffs always occurring when multiple columns exist ([#2686](#2686)) ([3275ad4](3275ad4)) * Fix issues [#2679](#2679) [#2721](#2721) ([#2723](#2723)) ([b0c9dd4](b0c9dd4)) * Fix several small issues ([#2697](#2697)) ([e3f6a15](e3f6a15)) * granting ownership on database roles ([#2703](#2703)) ([88944e7](88944e7)) * network policy update ([#2647](#2647)) ([8126b28](8126b28)) * Rename grants_redesign_desgin_decisions.md to grants_redesign_design_… ([#2691](#2691)) ([5000b2b](5000b2b)) * renames in update operations ([#2702](#2702)) ([793c879](793c879)) * showbyid method tests ([#2648](#2648)) ([ff5e617](ff5e617)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: snowflake-release-please[bot] <105954990+snowflake-release-please[bot]@users.noreply.github.com>

sfc-gh-asawicki · 2024-04-18T13:07:17Z

Hey @bmhah. We have released the new version: https://github.com/Snowflake-Labs/terraform-provider-snowflake/releases/tag/v0.89.0. Please bump and check if the error persists. Please remember to use the migration guide during version upgrades: https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/MIGRATION_GUIDE.md#migration-guide.

bmhah · 2024-04-18T15:06:46Z

Sounds good, we'll get to testing this later this week. I'll keep this issue open in the meantime

sfc-gh-asawicki · 2024-05-15T08:24:48Z

Hey @bmhah. Have you have a chance to test the newer versions after the introduced fixes?

bmhah · 2024-06-18T16:05:05Z

Yes, we've done the provider upgrade to 88 and have also used others, they haven't worked for us, the stage for this pipe still drops and recreates causing the pipe to break again

sfc-gh-asawicki · 2024-06-19T09:30:21Z

The fix was introduced in 0.89, not 0.88, so please provide the config and logs (run terraform with TF_LOG=DEBUG) on the 0.89+ version (I could not reproduce the issue given your config on the 0.89 version).

guillaumelecerf · 2024-06-21T15:19:46Z

We were having the same issue with:

file_format         = "FIELD_DELIMITER = '|' PARSE_HEADER = true"

Fix in #2885

We are getting permadiff on stage `file_format` parameter when it contains quotes, as it used to be the case with `copy_options` (see #2679) To fix it, re-using the same `DiffSuppressFunc` seems to do the trick. ## Test Plan * [x] acceptance tests * [x] locally built tested

sfc-gh-asawicki · 2024-07-11T08:45:14Z

The fix was included in v0.93.0. Please check it @guillaumelecerf @bmhah.

harukitake · 2024-07-17T07:28:36Z

Hey @sfc-gh-asawicki.
I have the same issue on the version 0.93.0.

terraform configuration

resource "snowflake_stage" "stage" {
      database = "DATABASE_NAME"
      file_format = "TYPE = CSV NULL_IF = []"
      name = "NAME"
      schema = "SCHEMA_NAME"
      storage_integration = "INTEGRATION_NAME"
      url = "s3://aaaaa"
    }

sfc-gh-asawicki · 2024-07-17T07:50:18Z

I will take a look this week. Thanks for providing the config; it helps reproduce the issues.

harukitake · 2024-09-30T01:43:56Z

Hi @sfc-gh-asawicki.
Do you have any updates for this issue?

sfc-gh-asawicki · 2024-09-30T15:12:21Z

Hey @harukitake. Sorry, I missed this one, I will check it this week.

sfc-gh-asawicki · 2024-10-10T09:29:30Z

Hey @harukitake. I've finally reproduced the error.

The current implementation of the stage resource filters out all the file format properties that have default values in Snowflake. TYPE = CSV is the default value, so after reading the resource's state, it is programmatically removed from the file format string, which causes the permadiff you see.

This is, of course, not the expected behavior. For now, please just remove TYPE = CSV from your config, because this is the default in Snowflake.

We will fix the handling of these parameters during the stage resource rework in the next few weeks. I have added a test to confirm it and the note in the resource documentation.

- Prove #2679; documentation added, fix will be done with the stage resource rework - Try to prove #3117; could not reproduce the error, test left for now

bmhah added the bug Used to mark issues with provider's incorrect behavior label Apr 5, 2024

sfc-gh-asawicki added a commit that referenced this issue Apr 18, 2024

Fix stage copy options permadiff

798a399

References: #2679

sfc-gh-asawicki added a commit that referenced this issue Apr 18, 2024

fix: Fix issues #2679 #2721 (#2723)

b0c9dd4

- Fix number of allowed values in tags - Suppress diff for stages `on_error`

snowflake-release-please bot mentioned this issue Apr 18, 2024

chore(main): release 0.89.0 #2692

Merged

sfc-gh-jcieslak added resource:stage Issue connected to the snowflake_stage resource category:resource labels May 20, 2024

guillaumelecerf mentioned this issue Jun 21, 2024

fix: use suppressQuoting to fix stage file_format permadiff #2885

Merged

2 tasks

sfc-gh-asawicki added a commit that referenced this issue Oct 10, 2024

Prove #2679 permadiff

92afd57

sfc-gh-asawicki mentioned this issue Oct 10, 2024

fix: Try to reproduce 2679 and 3117 #3124

Merged

sfc-gh-asawicki added a commit that referenced this issue Oct 10, 2024

fix: Try to reproduce 2679 and 3117 (#3124)

ccdbc30

- Prove #2679; documentation added, fix will be done with the stage resource rework - Try to prove #3117; could not reproduce the error, test left for now

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

snowflake_stage recreates every apply regardless of an actual change #2679

snowflake_stage recreates every apply regardless of an actual change #2679

bmhah commented Apr 5, 2024

sfc-gh-asawicki commented Apr 8, 2024

sfc-gh-bhill commented Apr 15, 2024 •

edited

Loading

bmhah commented Apr 17, 2024

sfc-gh-asawicki commented Apr 17, 2024

sfc-gh-asawicki commented Apr 18, 2024

bmhah commented Apr 18, 2024

sfc-gh-asawicki commented May 15, 2024

bmhah commented Jun 18, 2024

sfc-gh-asawicki commented Jun 19, 2024

guillaumelecerf commented Jun 21, 2024

sfc-gh-asawicki commented Jul 11, 2024

harukitake commented Jul 17, 2024

sfc-gh-asawicki commented Jul 17, 2024

harukitake commented Sep 30, 2024

sfc-gh-asawicki commented Sep 30, 2024

sfc-gh-asawicki commented Oct 10, 2024

snowflake_stage recreates every apply regardless of an actual change #2679

snowflake_stage recreates every apply regardless of an actual change #2679

Comments

bmhah commented Apr 5, 2024

Terraform CLI and Provider Versions

Terraform Configuration

Expected Behavior

Actual Behavior

Steps to Reproduce

How much impact is this issue causing?

Logs

Additional Information

sfc-gh-asawicki commented Apr 8, 2024

sfc-gh-bhill commented Apr 15, 2024 • edited Loading

bmhah commented Apr 17, 2024

sfc-gh-asawicki commented Apr 17, 2024

sfc-gh-asawicki commented Apr 18, 2024

bmhah commented Apr 18, 2024

sfc-gh-asawicki commented May 15, 2024

bmhah commented Jun 18, 2024

sfc-gh-asawicki commented Jun 19, 2024

guillaumelecerf commented Jun 21, 2024

sfc-gh-asawicki commented Jul 11, 2024

harukitake commented Jul 17, 2024

sfc-gh-asawicki commented Jul 17, 2024

harukitake commented Sep 30, 2024

sfc-gh-asawicki commented Sep 30, 2024

sfc-gh-asawicki commented Oct 10, 2024

sfc-gh-bhill commented Apr 15, 2024 •

edited

Loading