
May 9th, 2003, 09:17 AM
|
|
Junior Member
|
|
Join Date: May 2003
Posts: 1
Time spent in forums: < 1 sec
Reputation Power: 0
|
|
|
Perl Question - make sure array has unique strings only
I need to build a list (array) of urls in string form. Im going to incrementally add urls to the list as i crawl the web and find new urls to add. My question is how can i ensure that each time I am aobut to add a new url to the list, that it is not already in the list?
The only way i can think of, is to loop through the list before each add to make sure the newly found url is not already in there, but this must be really inefficient, especially since i expect my list to grow to 10000 in size or perhaps more.
Is there a better way to keep track of the urls ive alreay visited and avoid visiting them again?
|