Pipe question

13 posts / 0 new

Last post

Fri, 2015-04-17 12:21

trixie

Offline

Last seen: 7 months 4 days ago

Joined: 2011-02-03 13:58

Pipe question

I have a program that uses a plugin system. The main executable feeds audio data to the selected plugin sequentially. Whenever a chunk of data is received, the plugin invokes an encoding routine, which then writes the encoded portion of data into a file. The result is an MP3, FLAC etc. file, depending on the selected plugin.

I'd like to develop a new plugin that will not contain an encoding routine but, instead, will use an external encoder like Lame or ffmpeg to do the job. SoundFX (the old audio processing software) does that using a pipe which feeds data to Lame (for MP3) or Pegase (for MP2).

I've never used a pipe before, and I'd be very grateful for any tips / suggestions / caveats / code examples concerning the implementation using the AmigaDOS PIPE: Thanks!

Fri, 2015-04-17 16:20

broadblues

Offline

Last seen: 4 years 9 months ago

Joined: 2012-05-02 21:48

SoundFX (the old audio

SoundFX (the old audio processing software) does that using a pipe which feeds data to Lame (for MP3) or Pegase (for MP2).

Ugh how unix :-) Didn't realise it did that thought it had internal (and broken) encoders for those formats, might have to revue my SFX setup, as that could be useful...

Heres is how abc-shell handles pipes.

int pipe(int filedes[2])
{
        char pipe_name[1024];
 
#ifdef USE_TEMPFILES
#if defined(__amigaos4__)
        snprintf(pipe_name, sizeof(pipe_name), "/T/%x.%08x",
                 pipenum++,((struct Process*)FindTask(NULL))->pr_ProcessID);
#else
        snprintf(pipe_name, sizeof(pipe_name), "/T/%x.%08x",
                 pipenum++,GetUniqueID());
#endif
#else
#if defined(__amigaos4__)
        snprintf(pipe_name, sizeof(pipe_name), "/PIPE/%x%lu/32768/0",
                 pipenum++,((struct Process*)FindTask(NULL))->pr_ProcessID);
#else
        snprintf(pipe_name, sizeof(pipe_name), "/PIPE/%x%08x/4096/0",
                 pipenum++,GetUniqueID());
#endif
#endif
 
        filedes[1] = open(pipe_name, O_WRONLY|O_CREAT);
        filedes[0] = open(pipe_name, O_RDONLY);
 
    if (filedes[0] == -1 || filedes[1] == -1)
        {
                if (filedes[0] != -1)
                    close(filedes[0]);
                if (filedes[1] != -1)
                    close(filedes[1]);
 
                FUNCX;
                return -1;
        }
 
        FUNCX;
        return 0;
}

The readonly handle above becomes the stdin of your subprocess, and the parent writes data to the write end.

If you need to read data back then the the reverse is done, writeonly becomes stdout of subprocess and you read from the readonly end.

It's important to open the write end first (at least I've had issues with pipes blocking when I didn't).

That code is a bit of mess and you would most likely want to use DOS cxalls rather than lowlevel Clibrary file descriptor stuff. (abs-shell extracts the DOS handles later on)

One critical bit is the line

"/PIPE/%x%lu/32768/0"

translating that back into amiga

"PIPE:<name>/<buffersize>/<numbuffers>"

Specifying 0 for numbuffers makes the pipe dynamic, it will never fill (subject to available memory) and so you wont get hanging programs. Make buffersize larger if you have more larger amounts of data to pass through data.

Fri, 2015-04-17 22:16

Massi

Offline

Last seen: 4 years 6 months ago

Joined: 2012-03-28 17:16

@trixie I can' t remember of

@trixie

I can' t remember of any OS3 / OS4 specific source code examples, perhaps http://wiki.amigaos.net/wiki/Main_Page can give some hints.

Anyway "pipes" are a very common form of IPC (Inter Process Communication) from the Unix world. So it may be of help reading a Linux tutorial or such. The same concepts should apply to OS4.

Sun, 2015-04-19 11:44

Massi

Offline

Last seen: 4 years 6 months ago

Joined: 2012-03-28 17:16

@trixie Some more bits from

@trixie

Some more bits from my Linux experience, the same concepts should apply to OS4.

The synchronism between the communicating processes (2 or more) is based on the following:
1) a process gets blocked on trying to write data to a full pipe
2) a process gets blocked on trying to read data from an empty pipe

Pipes can only handle a limited amount of data (say 4096 bytes, see #define PIPE_BUF in limits.h), however the system guarantees the whole data transfer or nothing.

Pipes are very much suitable for applications based on Boss-Worker / Producer-Consumer model, where processes can easily added / removed at run time.

Mon, 2015-04-20 10:37

(Reply to #4) #5

trixie

Offline

Last seen: 7 months 4 days ago

Joined: 2011-02-03 13:58

Re: Pipe question

@broadblues

Ugh how unix :-)

It is, isn't it? :-D

FUNCX;

What does this actually do, or stand for?

@Massi
Thanks for the tips!

AmigaOne X5000-020 / 2GB RAM / Sapphire Pulse Radeon RX 560 / AmigaOS 4.1 Final Edition Update 2

Mon, 2015-04-20 20:37

(Reply to #5) #6

broadblues

Offline

Last seen: 4 years 9 months ago

Joined: 2012-05-02 21:48

@trixie just debug macro

@trixie just debug macro ignore it :-)

Thu, 2015-04-23 00:31

(Reply to #6) #7

xenic

Offline

Last seen: 2 years 7 months ago

Joined: 2011-05-07 04:52

@trixie Check out the

@trixie
Check out the queue.handler documentation in SYS:Documentation/handlers. The big problem might be getting lame etc. to work with an Amiga pipe.

X1000 - OS 4.1FE

Thu, 2015-04-23 12:45

(Reply to #7) #8

broadblues

Offline

Last seen: 4 years 9 months ago

Joined: 2012-05-02 21:48

The big problem might be

The big problem might be getting lame etc. to work with an Amiga pipe.

No problem there. Amiga pipes are quite functional, and so is lame.

9.AmigaOS4:> type Music:AudioEvolution4/Projects/08_04_30_Jazz_Tune/Jazztune_08_04_30.aiff | Music:Lame/lame  - >ram:out.mp3
LAME 3.98.2 32bits (http://www.mp3dev.org/)
Using polyphase lowpass filter, transition band: 16537 Hz - 17071 Hz
Encoding <stdin> to <stdout>
Encoding as 44.1 kHz j-stereo MPEG-1 Layer III (11x) 128 kbps qval=3

I can verify that ram:out.mp3 contained a viable mp3.

Thu, 2015-04-23 12:54

(Reply to #8) #9

broadblues

Offline

Last seen: 4 years 9 months ago

Joined: 2012-05-02 21:48

And for prgrams that don't

And for prgrams that don't support stdon/stdout but need a file name

9.AmigaOS4:> run >NIL: type Music:AudioEvolution4/Projects/08_04_30_Jazz_Tune/Jazztune_08_04_30.aiff >PIPE:foo/32768/0 
9.AmigaOS4:> Music:Lame/lame  PIPE:foo ram:out.mp3
LAME 3.98.2 32bits (http://www.mp3dev.org/)
Using polyphase lowpass filter, transition band: 16537 Hz - 17071 Hz
Encoding PIPE:foo to ram:out.mp3
Encoding as 44.1 kHz j-stereo MPEG-1 Layer III (11x) 128 kbps qval=3
    Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA 
  7995/7995  (100%)|    1:11/    1:11|    1:11/    1:11|   2.9341x|    0:00 
-------------------------------------------------------------------------------
   kbps        LR    MS  %     long switch short %                             
  128.0        2.8  97.2        97.2   1.7   1.1                               
Writing LAME Tag...done
ReplayGain: -4.0dB
9.AmigaOS4:>

Obviously you would add *>NIL: to deal with stderr output, or find the "quiet" switdh" I just left that out so show the commands were working.

Thu, 2015-04-23 16:44

(Reply to #9) #10

trixie

Offline

Last seen: 7 months 4 days ago

Joined: 2011-02-03 13:58

Re: Pipe question

@broadblues

Thanks! Now I only need to work out how to pipe the individual data portions so that lame receives a continuous stream representing the entire audio track.

AmigaOne X5000-020 / 2GB RAM / Sapphire Pulse Radeon RX 560 / AmigaOS 4.1 Final Edition Update 2

Sun, 2015-05-03 15:03

(Reply to #10) #11

ChrisH

Offline

Last seen: 4 years 3 months ago

Joined: 2010-12-07 20:22

I'm currently using a named

I'm currently using a named pipe (along with Curl) to handle httpS network connections. I'd be very interested to know if it's possible to have ANONYMOUS pipes on the Amiga. i.e. Connect two programs together, without having to give a specific "file" name to Pipe:.

Currently I generate a "random" 8-digit hexadecimal name, and hope that it's unique.

Author of the PortablE programming language.

I love using Amiga OS4.1 on my X1000 & Sam440 :-D

Sun, 2015-05-03 21:02

(Reply to #11) #12

broadblues

Offline

Last seen: 4 years 9 months ago

Joined: 2012-05-02 21:48

Ofcourse you can open an

Ofcourse you can open an anonymous pipe:

as this short shell session indicates:

New Shell process 9
9.AmigaOS4:> echo foo >PIPE:
9.AmigaOS4:> type PIPE:
foo
9.AmigaOS4:>

But anonymous pipes like that are only suitable for use in shell sessions where you can be reasonably guarenteed that no other program is going to try and open an anonymous pipe in te gap between you creating the write end and the read end.

To be sure you are opening your own pipe you *want* to use a named pipe.

But if you don't want to 'think' of your own name version 50 offers the UNIQUE switch.

/UNIQUE - Ignore the channel name and pick a unique name for this
pipe instead. This is useful for applications which pipe
input from one program to another. You create the pipe with
the /UNIQUE parameter, query the pipe's name using the
ExamineObject() function and then open the other pipe using the
name obtained (V50).

Mon, 2015-05-04 11:22

(Reply to #12) #13

ChrisH

Offline

Last seen: 4 years 3 months ago

Joined: 2010-12-07 20:22

But anonymous pipes like that

But anonymous pipes like that are only suitable for use in shell sessions where you can be reasonably guarenteed that no other program is going to try and open an anonymous pipe in te gap between you creating the write end and the read end.

OK, thanks. I wasn't sure what would happen if I (successfully) started reading + writing anonymously using a pipe, and then in the middle of using it another program started using another anonymous pipe. From what you say, it will get an entirely separate/new anonymous pipe :-) .

BTW, it appears not possible to see what (named) pipes already exist? Otherwise I could easily generate my own non-colliding name...

Author of the PortablE programming language.

I love using Amiga OS4.1 on my X1000 & Sam440 :-D

Search form

User login