Uploading an Excel with Data Shuttle Containing Duplicate Rows
I have a Data Upload workflow via Data Shuttle that runs on attachment for the most recent file. The excel files that stakeholders are uploading usually only have ~10 rows and when it goes through the workflow, the rows self-populate into the correct positions within a Smartsheet master database. The problem I have encountered is that sometimes the data being uploaded to Data Shuttle could contain a duplicate row that already exists within the Smartsheet. I want duplicates to be added to the bottom of the sheet like the rest of the new data already is, regardless of whether it is a duplicate or not. When there is a duplicate, Smartsheet doesn’t import that row into the sheet. Please let me know if this makes sense and any suggestions if you have them would be greatly appreciated.
Answers
-
Kleerfyre ✭✭✭✭✭✭Each row will have to have a unique identifier for it to work correctly. Is it possible for you to convert this process into your users using a form so that it will always add an new entry into your sheet?
Jonathan Sanders, CSM
"Change is always scary because it is unknown, but facing the unknown is what makes us stronger."
-
cko ✭
As of right now, our stakeholders use a form to upload the data into the sheet for Data Shuttle to do its thing. We are still in the preliminary phases of planning out this project but given the frequency of data being uploaded and the need for automation, we are really trying to stick to a model where an excel is uploaded (rows being auto populated into the Smartsheet database regardless if they are duplicates or not) and the rest taken care of. Someone from in my company suggested to look into if there is a way to not designate a unique record column but this doesn't seem to be the case. The only other thing I could think of would be to create a new Smartsheet sheet for every data upload but I don't think this is even feasible given the fact that many other sheets are linked to this main sheet via formulas.
-
Kleerfyre ✭✭✭✭✭✭
Why not just cut out the Data Shuttle part? You could use the form for the stakeholders to fill out instead of them typing it into an excel then clicking on a form to upload the excel to the sheet to have data shuttle grab it and then put it on the sheet.
Jonathan Sanders, CSM
"Change is always scary because it is unknown, but facing the unknown is what makes us stronger."
-
cko ✭
I wish I could cut Data Shuttle out of the picture. Unfortunately Data Shuttle has to be used because there is a separate database kept in another system used by a different company that generates an excel file for us to upload into our system.
-
Lesa Weaver ✭✭✭
HELLO@cko,
I had an issue with duplicate unique identifiers that I solved by having a column in my Excel spreadsheet that concatenates two columns to form a unique identifier column that is better for my Smartsheet. You could use a formula combining your current unique identifier + today so that it would always be unique within that day when it is uploaded. If you have duplicates within one day, you could add time.
In Smartsheet, I don't need the unique identifier primary column anymore after upload because I have the data in separate columns already, so I just hide that column.
Hope this helps.
Categories
<\/p>
Attempt two: can you update your trigger into \"When Rows Are Changed\"<\/p>
![\"image.png\"](\"https:\/\/us.v-cdn.net\/6031209\/uploads\/5Q6HHHJHRFET\/image.png\")
<\/img><\/a>\n <\/div>\n<\/div>\nThen, delete all of these erroneous rows and hit save.<\/p>
Finally, add \"No Show\" into like three rows. Save your sheet, and reload (automation takes a second to run completely). <\/p>
Is your \"Attendance\" column set up as a dropdown or as a text?<\/p>"}]}},"status":{"statusID":3,"name":"Accepted","state":"closed","recordType":"discussion","recordSubType":"question"},"bookmarked":false,"unread":false,"category":{"categoryID":321,"name":"Smartsheet Basics","url":"https:\/\/community.smartsheet.com\/categories\/smartsheet-basics%2B","allowedDiscussionTypes":[]},"reactions":[{"tagID":3,"urlcode":"Promote","name":"Promote","class":"Positive","hasReacted":false,"reactionValue":5,"count":0},{"tagID":5,"urlcode":"Insightful","name":"Insightful","class":"Positive","hasReacted":false,"reactionValue":1,"count":0},{"tagID":11,"urlcode":"Up","name":"Vote Up","class":"Positive","hasReacted":false,"reactionValue":1,"count":0},{"tagID":13,"urlcode":"Awesome","name":"Awesome","class":"Positive","hasReacted":false,"reactionValue":1,"count":0}],"tags":[]},{"discussionID":112207,"type":"question","name":"access to sheet","excerpt":"I want specific fields but I don't want a link to the sheet.... If someone has the link to the sheet would they even be able to access it?","snippet":"I want specific fields but I don't want a link to the sheet.... If someone has the link to the sheet would they even be able to access it?","categoryID":321,"dateInserted":"2023-10-26T15:25:53+00:00","dateUpdated":null,"dateLastComment":"2023-10-26T16:09:50+00:00","insertUserID":166642,"insertUser":{"userID":166642,"name":"JenaW","url":"https:\/\/community.smartsheet.com\/profile\/JenaW","photoUrl":"https:\/\/us.v-cdn.net\/6031209\/uploads\/defaultavatar\/nWRMFRX6I99I6.jpg","dateLastActive":"2023-10-26T17:45:01+00:00","banned":0,"punished":0,"private":false,"label":"✭✭"},"updateUserID":null,"lastUserID":142964,"lastUser":{"userID":142964,"name":"Ray Lindstrom","title":"","url":"https:\/\/community.smartsheet.com\/profile\/Ray%20Lindstrom","photoUrl":"https:\/\/us.v-cdn.net\/6031209\/uploads\/userpics\/5Y0KP6UKIBTU\/nY3615V0DPRZV.jpg","dateLastActive":"2023-10-26T16:55:30+00:00","banned":0,"punished":0,"private":false,"label":"✭✭✭✭✭✭"},"pinned":false,"pinLocation":null,"closed":false,"sink":false,"countComments":3,"countViews":33,"score":null,"hot":3396672343,"url":"https:\/\/community.smartsheet.com\/discussion\/112207\/access-to-sheet","canonicalUrl":"https:\/\/community.smartsheet.com\/discussion\/112207\/access-to-sheet","format":"Rich","lastPost":{"discussionID":112207,"commentID":401859,"name":"Re: access to sheet","url":"https:\/\/community.smartsheet.com\/discussion\/comment\/401859#Comment_401859","dateInserted":"2023-10-26T16:09:50+00:00","insertUserID":142964,"insertUser":{"userID":142964,"name":"Ray Lindstrom","title":"","url":"https:\/\/community.smartsheet.com\/profile\/Ray%20Lindstrom","photoUrl":"https:\/\/us.v-cdn.net\/6031209\/uploads\/userpics\/5Y0KP6UKIBTU\/nY3615V0DPRZV.jpg","dateLastActive":"2023-10-26T16:55:30+00:00","banned":0,"punished":0,"private":false,"label":"✭✭✭✭✭✭"}},"breadcrumbs":[{"name":"Home","url":"https:\/\/community.smartsheet.com\/"},{"name":"Get Help","url":"https:\/\/community.smartsheet.com\/categories\/get-help"},{"name":"Smartsheet Basics","url":"https:\/\/community.smartsheet.com\/categories\/smartsheet-basics%2B"}],"groupID":null,"statusID":3,"image":{"url":"https:\/\/us.v-cdn.net\/6031209\/uploads\/GCNDC3R4VSAE\/h.png","urlSrcSet":{"10":"https:\/\/us.v-cdn.net\/cdn-cgi\/image\/fit=scale-down,width=10\/https:\/\/us.v-cdn.net\/6031209\/uploads\/GCNDC3R4VSAE\/h.png","300":"https:\/\/us.v-cdn.net\/cdn-cgi\/image\/fit=scale-down,width=300\/https:\/\/us.v-cdn.net\/6031209\/uploads\/GCNDC3R4VSAE\/h.png","800":"https:\/\/us.v-cdn.net\/cdn-cgi\/image\/fit=scale-down,width=800\/https:\/\/us.v-cdn.net\/6031209\/uploads\/GCNDC3R4VSAE\/h.png","1200":"https:\/\/us.v-cdn.net\/cdn-cgi\/image\/fit=scale-down,width=1200\/https:\/\/us.v-cdn.net\/6031209\/uploads\/GCNDC3R4VSAE\/h.png","1600":"https:\/\/us.v-cdn.net\/cdn-cgi\/image\/fit=scale-down,width=1600\/https:\/\/us.v-cdn.net\/6031209\/uploads\/GCNDC3R4VSAE\/h.png"},"alt":"h.png"},"attributes":{"question":{"status":"accepted","dateAccepted":"2023-10-26T16:01:38+00:00","dateAnswered":"2023-10-26T15:58:49+00:00","acceptedAnswers":[{"commentID":401852,"body":"
Hi @JenaW<\/a>,<\/p> You have the option of using \"Message only\", and then displaying your two fields within the message using double curly brackets like this:<\/p> The tradeshow named {{Association\/Tradeshow Name}} will be held on {{Tradeshow Date}}.<\/p>"}]}},"status":{"statusID":3,"name":"Accepted","state":"closed","recordType":"discussion","recordSubType":"question"},"bookmarked":false,"unread":false,"category":{"categoryID":321,"name":"Smartsheet Basics","url":"https:\/\/community.smartsheet.com\/categories\/smartsheet-basics%2B","allowedDiscussionTypes":[]},"reactions":[{"tagID":3,"urlcode":"Promote","name":"Promote","class":"Positive","hasReacted":false,"reactionValue":5,"count":0},{"tagID":5,"urlcode":"Insightful","name":"Insightful","class":"Positive","hasReacted":false,"reactionValue":1,"count":0},{"tagID":11,"urlcode":"Up","name":"Vote Up","class":"Positive","hasReacted":false,"reactionValue":1,"count":0},{"tagID":13,"urlcode":"Awesome","name":"Awesome","class":"Positive","hasReacted":false,"reactionValue":1,"count":0}],"tags":[]},{"discussionID":112167,"type":"question","name":"After approval, update request","excerpt":"1 week after approval, I need an update request to go out. Is that possible? How can this automation be set up?","snippet":"1 week after approval, I need an update request to go out. Is that possible? How can this automation be set up?","categoryID":321,"dateInserted":"2023-10-25T21:01:48+00:00","dateUpdated":null,"dateLastComment":"2023-10-26T14:47:56+00:00","insertUserID":166642,"insertUser":{"userID":166642,"name":"JenaW","url":"https:\/\/community.smartsheet.com\/profile\/JenaW","photoUrl":"https:\/\/us.v-cdn.net\/6031209\/uploads\/defaultavatar\/nWRMFRX6I99I6.jpg","dateLastActive":"2023-10-26T17:45:01+00:00","banned":0,"punished":0,"private":false,"label":"✭✭"},"updateUserID":null,"lastUserID":166642,"lastUser":{"userID":166642,"name":"JenaW","url":"https:\/\/community.smartsheet.com\/profile\/JenaW","photoUrl":"https:\/\/us.v-cdn.net\/6031209\/uploads\/defaultavatar\/nWRMFRX6I99I6.jpg","dateLastActive":"2023-10-26T17:45:01+00:00","banned":0,"punished":0,"private":false,"label":"✭✭"},"pinned":false,"pinLocation":null,"closed":false,"sink":false,"countComments":4,"countViews":35,"score":null,"hot":3396601784,"url":"https:\/\/community.smartsheet.com\/discussion\/112167\/after-approval-update-request","canonicalUrl":"https:\/\/community.smartsheet.com\/discussion\/112167\/after-approval-update-request","format":"Rich","lastPost":{"discussionID":112167,"commentID":401829,"name":"Re: After approval, update request","url":"https:\/\/community.smartsheet.com\/discussion\/comment\/401829#Comment_401829","dateInserted":"2023-10-26T14:47:56+00:00","insertUserID":166642,"insertUser":{"userID":166642,"name":"JenaW","url":"https:\/\/community.smartsheet.com\/profile\/JenaW","photoUrl":"https:\/\/us.v-cdn.net\/6031209\/uploads\/defaultavatar\/nWRMFRX6I99I6.jpg","dateLastActive":"2023-10-26T17:45:01+00:00","banned":0,"punished":0,"private":false,"label":"✭✭"}},"breadcrumbs":[{"name":"Home","url":"https:\/\/community.smartsheet.com\/"},{"name":"Get Help","url":"https:\/\/community.smartsheet.com\/categories\/get-help"},{"name":"Smartsheet Basics","url":"https:\/\/community.smartsheet.com\/categories\/smartsheet-basics%2B"}],"groupID":null,"statusID":3,"attributes":{"question":{"status":"accepted","dateAccepted":"2023-10-26T21:35:00+00:00","dateAnswered":"2023-10-26T14:42:15+00:00","acceptedAnswers":[{"commentID":401825,"body":"
