Skip to content

Instantly share code, notes, and snippets.

@damiththa
Last active December 11, 2019 18:14
Show Gist options
  • Save damiththa/ea4aae5eec68c7698785e8d710230e19 to your computer and use it in GitHub Desktop.
Save damiththa/ea4aae5eec68c7698785e8d710230e19 to your computer and use it in GitHub Desktop.
Looping over a list while conditon is met

‎‎Looping over a list

Imagine you have a list where you then need to take a set portion of it as a batch.

Below gits shows how to use a while loop for this to creat batches.

By converting two lists into sets and then comparing them, we can get a list of items that weren't batch first time around. This works for a regular list.

However for a list of json objects, as above, we can't use set. If used as a regular list, it throws below error.

TypeError: unhashable type: 'dict'

and the reason is, as nicely described here

You're trying to use a dict as a key to another dict or in a set. That does not work because the keys have to be hashable. As a general rule, only immutable objects (strings, integers, floats, frozensets, tuples of immutables) are hashable (though exceptions are possible).

Therefore, to do the same, we could take advantage of slice as shown in the second code gist.

def main():
batch_cutoff = 10
# ------ NOTEME: simple python list with items --------
msgEntries_sendBatch = []
# msgEntries = [1,3,4,5,7,6,40,2,60,70,9,30,10,8,48,22,0,45] # 18 in the list
msgEntries = [1,3,4,5,7,8,9,2,6,0] # 10 in the list i.e. same as the cutoff no.
# msgEntries = [1,3,4,5,7] # 5 in the list
# msgEntries = [] # 0 in the list
while len(msgEntries) != 0:
msgEntries_sendBatch = msgEntries[:batch_cutoff]
print msgEntries_sendBatch
msgEntries = list(set(msgEntries) - set(msgEntries_sendBatch))
print len(msgEntries)
print 'came here now'
print '-------------------------------------------'
if __name__ == "__main__":
main()
def main():
batch_cutoff_json = 2
msgEntries_sendBatch_json = []
msgEntries_holdBatch_json = []
msgEntries_json = [
{'Id': '9a21f0ff-8e02-4870-a3dc-cd9495452b0b', 'MessageBody': 'This MiSeq step is ready for FusionQC.',
'MessageAttributes': {'clarityLIMS_Instance': {u'DataType': 'String', u'StringValue': 'PROD'}, 'stepID': {u'DataType': 'String', u'StringValue': '24-15005'}}
},
{'Id': 'b1ad4ad4-e83e-43b9-9b2e-3c6ab37fbde4', 'MessageBody': 'This MiSeq step is ready for FusionQC.',
'MessageAttributes': {'clarityLIMS_Instance': {u'DataType': 'String', u'StringValue': 'PROD'}, 'stepID': {u'DataType': 'String', u'StringValue': '24-14030'}}
},
{'Id': '9a21f0ff-8e02-4870-a3dc-cd94ee452b0b', 'MessageBody': 'This MiSeq step is ready for FusionQC.',
'MessageAttributes': {'clarityLIMS_Instance': {u'DataType': 'String', u'StringValue': 'PROD'}, 'stepID': {u'DataType': 'String', u'StringValue': '24-150505'}}
},
{'Id': 'b1aq4ad4-e83e-4w9-9b2e-3c6ab37fbde4', 'MessageBody': 'This MiSeq step is ready for FusionQC.',
'MessageAttributes': {'clarityLIMS_Instance': {u'DataType': 'String', u'StringValue': 'PROD'}, 'stepID': {u'DataType': 'String', u'StringValue': '24-14036'}}
}
]
while len(msgEntries_json) != 0:
msgEntries_sendBatch_json = msgEntries_json[:batch_cutoff_json]
print msgEntries_sendBatch_json
msgEntries_holdBatch_json = msgEntries_json[batch_cutoff_json:]
print msgEntries_holdBatch_json
msgEntries_json = msgEntries_holdBatch_json
print len(msgEntries_json)
print 'came here now - json'
if __name__ == "__main__":
main()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment