​@James Luterek ​@Patrick Burkart ​@Obed Murillo ​@Philipp Hofmann are you guys familiar with this topic?  Please give Alex a hand if you can! Thanks in advance!  

Hi Alex,

long story short: Been there, done that!

We had the exact same issue - our PO wanted to blacklist products in some markets manually using the Merchant Center. So we defined a field which would contain a list of blacklisted markets - and every import would have overwritten this.

Commercetools has no way of knowing which parts of the product not to overwrite I guess. Of course, values which are not defined in the import objects might be just be kept, but this makes it hard to delete something via the import if that was implemented.

Our import fetches all products, then creates the import objects and then enriches the values which should be copied to the reimported products. Then off it goes into the import containers.

We’ve made quite good experience with this actually. It’s not that complicated to implement either.

I’ve had a quick look at the documentation and found no updates concerning this (we implemented that about 1.5 years ago). 

Bottom line: Our guts are in sync. But I can’t think of a way how commercetools would know what to keep and what to delete. Therefore, we stuck to the way you already suggested as well.

Do you end up with race conditions? Since the import API is asynchronous, it’s technically possible for a user to manually change a value while the import queue is being processed having their manual change overwritten.

When dealing with data both imported and managed through the Merchant Center, I tend to fall back to the product APIs instead of the import APIs. It requires more control in throttling and optimization, but the update actions give full control over which fields are impacted with each call.

Nope, we did not end up with race conditions. Mainly though because our import ran at night, and our PO didn’t change anything at night (I know, we’re professionals….)

Jokes aside, you raise a very valid point which should be kept in mind.

I guess this depends on the use case. For us, this race condition was so acceptable that we didn’t spend much time evaluating it. 

Hi ​@Alex 

We had similar requirement on our project, there were products that are enriched in MC but if an update comes from the importAPI will override, so what we did is the following:

Extension was created with the following trigger:

      "triggers": s
        {
          "resourceTypeId": "product",
          "actions": s
            "Create",
            "Update"
          ],
          "condition": "masterData(current(masterVariant(attributes(name=\"customAttribute\" and value=\"SPECIAL_ATTRIBUTE_VALUE\"))))"
        }
      ],
      "key": "prevent-protected-fields-update"

As you can see this will trigger on create and update, and it will happen only for the condition you define, in this example we are having this condition to execute only on a particular set of items, not on the whole catalog.

Then the Serverless function that does this basically have a list of attributes that we are protecting to be updated from the ImportAPI.

This worked fine for use given the products that have this particular rule are a small set, but if your set of products is not small this will not fly since it will impact performance and I’ll suggest to try to find another alternative.

I hope this other alternative give you more input to solve your problem.

Regards,

Obed

​@Alex 
A disclaimer about this, the Product Extension is on beta, if you see the documentation: https://docs.commercetools.com/api/projects/api-extensions you can see Products are not there.

So this is something you might consider when doing your development.

Regards,

Obed

Thanks ​@Patrick Burkart  ​@Obed Murillo ​@James Luterek for your input. It’s very valuable and it helps me a lot moving forward.

We have around 30k products, so having the Product Extension triggering nightly for each one is a bit too much I guess . But it’s definitely good to know about it!

My case is similar to Patrick’s. The PO will for sure not update those attributes during the night when the import runs so we are not worried about any race conditions. Also, they are planning on using the Product Import in the Merchant Center to do a bulk update every once in a while, but also have the possibility to manually update the value for a specific product and be able to see and filter for it in the product catalog. Basically the information in commercetools is the golden record at the time when the import is running and it should be maintained.

I was thinking on storing the key-value pair in a custom object and get this list instead of querying the products directly during the import, for performance considerations. It should work well, but I don’t have any trigger on the import containers to initiate an update of the custom object. I was hoping this way I can get the file used and update the custom object to keep it in sync. Obed’s suggestion with the product extension could be one way, but the catalog will be updated in bulk at some point, so I’m worried about the excessiveness of the solution.

I guess I will take the beaten path and do a product query before the import.

Your help is much appreciated!

Cheers all,
Alex

Hi Alex, 

You can follow below approach to solve this: 

1. When creating a new product & variants, use subscriptions to create an alert where you can notify the business users to update this attribute in MC or via API. Or if the team has any other alternative ways to find out the need to set this attribute upon product creation, please continue to utilize that. 
2. For subsequent updates on products via import API, use product variant patch option which allows updating or removing an existing attribute. This option allows delta updates where you don’t need to send an existing attribute value in the input unless you need to modify it’s value or remove the value. 

Hi Hetvi,

Thanks for your input.

I was contemplating the product variant patch, but that one would have required more refactoring than I was willing to do at this point in time.

I like the idea of automatically making the business users aware of any changes that might impact them, but in this case it is not viable, since it would mean a daily notification because our import runs every night. I’m pretty sure I would become their least favourite person.

Nevertheless, I’m happy to report I’ve successfully implemented today the approach that Patrick also suggested and it works fine.

Regards,
Alex

Thanks everyone who took part in this discussion and ​@Alex thanks for updating us about the results. Wishing everyone a great week!