Moving - not copying - data in Azure Data Factory

azure data factory v2 copy activity
azure data factory delete file after copy
azure data factory copy data overwrite
azure data factory dynamic file name
azure data factory copy multiple files
azure data factory activity output
azure data factory copy behavior merge files
azure data factory copy all files in folder

I'd like to set up an Azure Data Factory pipeline which performs a move (i.e. copy, verify, delete) operation rather than just a copy operation between Blob Storage and a Data Lake Store. I cannot seem to find any detail on how to do this.

Move files between file-based storage, Move files with Azure Data Factory that there will NOT be new files arriving into the folder between copying operation and deleting operation. KensaQ updates its results daily to help you find what you are looking for. Search Now! Find about microsoft azure on KensaQ.com.

Azure Data Factory does not have a built-in activity or option to Move files as opposed to Copy them. You can however do this with a Custom Activity.

This example on github shows how to do this with Azure Blob:

...
blob.DeleteIfExists();
...

https://github.com/Azure/Azure-DataFactory/tree/master/Samples/DeleteBlobFileFolderCustomActivity

If you feel this is an important feature, please add a feedback request:

https://feedback.azure.com/forums/270578-data-factory

A Delete activity has been added recently:

https://azure.microsoft.com/en-us/blog/clean-up-files-by-built-in-delete-activity-in-azure-data-factory/

Copy activity in Azure Data Factory, Note: This post is about Azure Data Factory V1 I've spent the last couple You can copy a file with a copy activity, but you cannot actually move� Search For Azure Cloud Service Fast and Save Time. Find Azure Cloud Service Here with us!

I Like to Move It, Move It – But Azure Data Factory Doesn't, Azure Data Factory does not have a built-in option for moving files but we can make use of copy operation instead. Create a new data factory. Find What Is Azure Cloud Service. Find Quick Results from Multiple Sources

Moving - not copying - data in Azure Data Factory, Azure Data Factory - Metadata Activity (Part 1) Azure Data Factory… In this demo we first Duration: 11:18 Posted: May 2, 2020 Azure Data Factory does not have a built-in option for moving files but we can make use of copy operation instead. Create a new data factory After deployment of data factory, click on Author and monitor Now select copy data

Azure Data Factory–Copy and Delete Activities, In this post, we dig into the Copy Data Activity in Azure Data Factory and look retry interval, and whether or not to log input and output details. To do that, we need to read the source data in CSV format and transfer it as bits� APPLIES TO: Azure Data Factory Azure Synapse Analytics (Preview) ADF copy activity has built-in support on “move” scenario when copying binary files between storage stores. The way to enable it is to set “deleteFilesAfterCompletion” as true in copy activity. By doing so, copy activity will delete files from data source store after job completion.

Copy Data Activity in Azure Data Factory, This tutorial will not start from creating an Azure Data Factory (ADF) Expand Move & transform section, and drag Copy data to the main area� In this tip, we’ve shown how you can copy data from Azure Blob storage to a table in a Snowflake database and vice versa using Azure Data Factory. At the moment, ADF only supports Snowflake in the Copy Data activity and in the Lookup activity, but this will be expanded in the future.

Comments
  • docs.microsoft.com/en-us/azure/data-factory/…
  • MS docs abuse "move" to mean "copy", I have of course read the link you posted.
  • In 2020, this should be the right answer. Microsoft also released a template for moving files at the end of 2019. I suggest adding it to your answer: docs.microsoft.com/en-us/azure/data-factory/…
  • Rather painful that this requires me to build a DLL...I'll give it a go though, thank you.
  • Consider making the feature request and post a link here. I'll vote for it, plus others landing at the page might too. Also if you get some code working, feel free to post it and mark as an answer yourself; I think that would be genuinely useful.