Hollywood 5 - Making use of the Network API

  • up
    39%
  • down
    61%

Hi,

Its been a while since my last blog post here at OS4Coding and I was hoping to start a series of tutorials, but due to time constraints and 'is it really worth it' and 'is anyone listening?' I finally decided to make another post for interested Hollywood developers.

In this blog I attach some source code for adding asynchronous downloads into your project(s).

This code was created out of a necessity to add support for asynchronous downloads into my application Jack.

I created the below code this weekend and hope to introduce it into Jack over the course of the next week and into next weekend.

The example code here will make a connection to OS4Depot and download 3 files simultaneously and save them to RAM.

C/C++ developers will notice straightaway its not very long, I can personally imagine doing the same thing in C/C++ would require much more work and this is the appeal of Hollywood to many users regardless of experience.

Foreword: This code is not optimised in any way and is purely my testing code before migrating it across into Jack where I can then add error handling (404 codes) and other such requirements of the project. This code is also particularly catered towards OS4Depot where HTTP Redirection is in place to reach the downloadable files- if you are going to use this code to download from elsewhere you may well have to do adapt the code accordingly; e.g. where HTTP redirection is not in place.

This example code allows for up to 30 concurrent downloads, but can easily be changed at a whim - and indeed I will no doubt reduce this to 10 or perhaps 15. I am positive Origin won't appreciate the server being overloaded!

I won't go into too much detail here in this blog on how it all works - suffice to say - if you have intermediate level of understanding of Hollywood it'll make perfect sense.

Basically we say which file we want to download using base:DownloadFile() and how big it is. The event handler 'OnReceiveData' spools the data in chunks of 65336 and writes them to disk, when the download is complete we close the connection and the file with SaveToDisk().

base:AllocateFindFilePort() is responsible for Allocating or Finding an available File Port an assigning it to the connection.

As all this is happening the program flow continues and the program can do whatever it needs to do between Repeat.. WaitEvent()... Forever.

Feel free to re-use or adapt my code as you see fit.

This completes my short blog on asynchronous downloading of files with Hollywood.

Here's the code...

  1. ; This asychronous download example code is adapted to OS4Depot where HTTP
  2. ; redirection is in place and must be adapted if the host site doesn't use
  3. ; HTTP redirection.
  4.  
  5. Global base, data$, count, done, host$
  6.  
  7. host$ = "www.os4depot.net"
  8.  
  9. base = {}
  10.  
  11. ; Userdata, FilePort, Remote Archive, Bytes Received, Total Filesize
  12. DimStr connections[30][2][1][1][1]
  13.  
  14. For Local c = 0 to 29
  15. connections[c][0] = ""
  16. connections[c][1] = c
  17. connections[c][2] = ""
  18. connections[c][3] = 0
  19. connections[c][4] = 0
  20. Next
  21.  
  22. Function base:AllocateFindFilePort(userdata)
  23.  
  24. Local c, available, found
  25.  
  26. If userdata = nil ; Allocate File Port
  27. For c = 0 to 29
  28. If GetType(connections[c][0]) <> #LIGHTUSERDATA
  29. available = True
  30. Break
  31. EndIf
  32. Next
  33.  
  34. ; If no connections available return Nil, maxconnections set in DimStr connections[]
  35. If available = False
  36. c = Nil
  37. EndIf
  38.  
  39. Else ; Find File Port via userdata
  40.  
  41. For c = 0 to 29
  42. If GetType(connections[c][0]) = #LIGHTUSERDATA
  43. If connections[c][0] = userdata
  44. found = True
  45. Break
  46. EndIf
  47. EndIf
  48. Next
  49.  
  50. If found = False
  51. c = Nil
  52. EndIf
  53. EndIf
  54.  
  55. Return (c)
  56. EndFunction
  57.  
  58. Function base:DownloadFile(server$, url$, downloadpath$, filename$, filesize)
  59.  
  60. Local id = base:AllocateFindFilePort()
  61.  
  62. Local data$, count, done
  63.  
  64. connections[id][0] = OpenConnection(nil, server$, 80)
  65.  
  66. SendData(connections[id][0], "GET " .. url$ .. " HTTP/1.0\r\n\r\n")
  67.  
  68. ; This line here will not work if the site doesn't use redirection, you will need to
  69. ; adapt this accordingly.
  70. data$, count, done = ReceiveData(connections[id][0], #RECEIVEALL)
  71.  
  72. ; Redirect
  73. If FindStr(data$,"302 Found") > -1
  74. CloseConnection(connections[id][0])
  75. redirection$ = MidStr(data$,FindStr(data$,"Location: "),StrLen(data$))
  76. redirection$ = MidStr(redirection$,10,FindStr(redirection$,"\n")-11)
  77.  
  78. server$ = ReplaceStr(redirection$, "http://", "")
  79. server$ = MidStr(server$,0,FindStr(server$,"/"))
  80.  
  81. connections[id][0] = OpenConnection(nil, server$, 80)
  82.  
  83. SendData(connections[id][0], "GET " .. redirection$ .. " HTTP/1.0\r\n\r\n")
  84.  
  85. data$ = nil
  86.  
  87. ; Skip HTTP Headers
  88. While data$ = nil Or FindStr(data$,"Content-Type:") = -1
  89. data$, count, done = ReceiveData(connections[id][0], #RECEIVELINE)
  90. Wend
  91.  
  92. data$, count, done = ReceiveData(connections[id][0], #RECEIVELINE) ; Skip blank line
  93.  
  94. ; Adapt this for sites that don't use HTTP rdirection
  95. Else
  96. )
  97. EndIf
  98.  
  99. ; Not Found
  100. If FindStr(data$,"404 Found") > -1
  101. Return(False)
  102.  
  103. ; Found - Download it
  104. Else
  105.  
  106. connections[id][1] = OpenFile(nil,downloadpath$ .. filename$, #MODE_READWRITE)
  107. connections[id][2] = filename$
  108. connections[id][4] = filesize
  109.  
  110. Return(True)
  111. EndIf
  112. EndFunction
  113.  
  114. Function base:BuildFile(userdata)
  115. Local segment$
  116. segment$, count, done = ReceiveData(userdata.id, #RECEIVEBYTES, 65536) ; Read file data
  117.  
  118. Local port = base:AllocateFindFilePort(userdata.id)
  119.  
  120. WriteString(connections[port][1],segment$)
  121.  
  122. connections[port][3] = connections[port][3] + StrLen(segment$)
  123.  
  124. If connections[port][3] = connections[port][4] Then SaveToDisk(userdata.id,port)
  125. EndFunction
  126.  
  127. Function base:Keyboard(msg)
  128. Switch msg.action
  129. Case "OnKeyDown"
  130. If msg.key = "ESC"
  131. End()
  132. EndIf
  133. EndSwitch
  134. EndFunction
  135.  
  136. Function base_BuildFile(msg) base:BuildFile(msg) EndFunction
  137. Function base_KeyboarD(msg) base:Keyboard(msg) EndFunction
  138.  
  139. InstallEventHandler({OnReceiveData = base_BuildFile, OnKeyDown = base_Keyboard})
  140.  
  141. debugprint("Asynchronous file download demo by Richard Lake (richard@lakemarketingandevents.co.uk)")
  142. debugprint("\nWorks with sites like OS4Depot where HTTP Redirect is in place.")
  143. debugprint("\nOne limitation in this demo is that you must know beforehand the filesize of each download in advance.")
  144. debugprint("\nDownloading 3 files from OS4Depot asynchronously...")
  145.  
  146. success = base:DownloadFile(host$,"http://" .. host$ .. "/share/game/board/africa.lha","RAM:","africa.lha",297517)
  147. success = base:DownloadFile(host$,"http://" .. host$ .. "/share/utility/misc/ipqalc.lha","RAM:","ipqalc.lha",114081)
  148. success = base:DownloadFile(host$,"http://" .. host$ .. "/share/emulation/gamesystem/amiarcadia.lha","RAM:","amiarcadia.lha",3454033)
  149.  
  150. Function SaveToDisk(userdata,port)
  151. CloseConnection(userdata)
  152. CloseFile(connections[port][1])
  153.  
  154. debugprint(connections[port][2] .. " saved to RAM:")
  155.  
  156. connections[port][0] = ""
  157. connections[port][2] = ""
  158. connections[port][3] = 0
  159. connections[port][4] = 0
  160. Endfunction
  161.  
  162. Repeat
  163. WaitEvent()
  164. Forever

Tags: 

Blog post type: 

Comments

Tedzogh's picture

Great blog to examin for a soon to be HW owner.
Thanks

stephenix1015's picture

This is just the tip of the iceberg. If you want to see some more, including function to develop AmigaOS-like applications complete with buttons, icon bars, option boxes and other event handling.-Missed Fortune