Create a sample code for generic use cases

[uttamgupta]

I have read document with pseudo code but it doesn't help. e.g. how to create serverKeyVerifier as a delegate. How to restrict connection to server when key doesn't exists in known_hosts file etc. It would be great if anyone provides full working example code.

[uttamgupta]

Somehow I am able execute command remotely but not able to run interactive. I have read this document -

---

Running an interactive command/shell

If one needs to parse the command/shell output and then respond by sending the correct input, the code must use separate thread(s) to read the STDOUT/STDERR and provide STDIN input. These threads must be up and running before opening the channel since data may start to pour in even before the await/verify call returns. If this data is not consumed at a reasonable pace, then channel may block and eventually even disconnect. Thus the thread(s) using the streams must be ready beforehand.

// The same code can be used when opening a ChannelExec in order to run a single interactive command
try (ClientChannel channel = session.createShellChannel(/* use internal defaults */)) {
channel.setIn(...stdin...);
channel.setOut(...stdout...);
channel.setErr(...stderr...);
...spawn the servicing thread(s)....
try {
channel.open().verify(...some timeout...);
// Wait (forever) for the channel to close - signalling shell exited
channel.waitFor(EnumSet.of(ClientChannelEvent.CLOSED), 0L);
} finally {
// ... stop the pumping threads ...
}
}
In such cases it is recommended to use the inverted streams in the relevant threads

// The same code can be used when opening a ChannelExec in order to run a single interactive command
try (ClientChannel channel = session.createShellChannel(/* use internal defaults */)) {
try {
channel.open().verify(...some timeout...);

    spawnStdinThread(channel.getInvertedIn());
    spawnStdoutThread(channel.getInvertedOut());
    spawnStderrThread(channel.getInvertedErr());

    // Wait (forever) for the channel to close - signalling shell exited
    channel.waitFor(EnumSet.of(ClientChannelEvent.CLOSED), 0L);
} finally {
    // ... stop the pumping threads ...
}

}

---

I couldn't understand how to read and write commands in different threads. It would be great If any one could write a sample code.
Basically my use is that, I want to run command. sudo rm "abc" it will wait to get password and I need to provide password programmatically without creating shell. How can I achieve this?

[tomaswolf]

Basically my use is that, I want to run command. su rm "abc" it will wait to get password and I need to provide password programmatically without creating shell. How can I achieve this?

I presume you mean sudo, not su.

First, be aware that Apache MINA sshd has no privilege separation, and is not tied into the Unix user management. The command will run with the user that is running the server.

Regarding sudo itself, I see two possibilities:

• Configure sudo on the server side such that the user can run this precise command (rm "abc") password-less, or
• Use sudo -S and pass in the password via stdin (channel.getInvertedIn()).

[uttamgupta]

Thanks for the response. I have tried with getInvertedIn(), but it was returning null. I am still trying, it would be great help if you already have sample code for this use case. That's the reason I have created this issue to provide sample code. btw I fixed my typo from "su" to "sudo". Another use case could be if I run binary calculator command "bc" then it waits for response from user e.g. 2*2 or 3+4 and keeps on going. How can I achieve this scenario?

[tomaswolf]

If you have a case where the sequence

ChannelExec cmd = session.createExecChannel("some command");
cmd.open().verify(SOME_TIMEOUT);
OutputStream stdin = cmd.getInvertedIn();

results in stdin == null, then please post the code you use to open the channel. Note that you should call getInvertedIn() after having openend the channel. (Replace "some command" by the command you want to run.)

[uttamgupta]

Thanks for helping me. stdin is not null this time, I was doing some mistakes and still doing some mistake. Here is the code -
public class InteractiveExecCmd {
public static void main(String[] args) throws Exception {

    SshClient client = SshClient.setUpDefaultClient();
    client.start();
    String hostname = "<hostIp>";
    String password = "<password>";
    ConnectFuture cf = client.connect("root", hostname, 22);
    ClientSession clientSession = cf.verify().getSession();
    clientSession.addPasswordIdentity(password);
    clientSession.auth().verify(TimeUnit.SECONDS.toMillis(30));
    ChannelExec ce;
    try {
        ce = clientSession.createExecChannel("bc");

       // ByteArrayOutputStream out = new ByteArrayOutputStream();
       // ByteArrayOutputStream err = new ByteArrayOutputStream();
       // ByteArrayInputStream inputStream = new ByteArrayInputStream(GenericUtils.EMPTY_BYTE_ARRAY);

        //ce.setOut(out);
        //ce.setErr(err);
        //ce.setIn(inputStream);
        OpenFuture openFuture = ce.open().verify(30, TimeUnit.SECONDS);
        Stack<String> cmds = new Stack<>();
        cmds.add("2*2");
        cmds.add("2+2");
        OutputStream stdin = ce.getInvertedIn();
        Thread t1 = new Thread(() -> {
            // Code to be executed in the new thread
            while (!cmds.empty()) {
                    try {
                        Thread.sleep(1000);
                    } catch (InterruptedException e) {
                        throw new RuntimeException(e);
                    }
                try {
                    stdin.write(cmds.pop().getBytes());
                    System.out.println("Out1 : " + ce.getOut().toString());
                    System.out.println("Out2 " + stdin.toString());
                    System.out.println("Out3 " + ce.getOut().toString());
                } catch (IOException e) {
                    throw new RuntimeException(e);
                }
            }
        });
        t1.start();
        t1.join();
        Set<ClientChannelEvent> events =
                ce.waitFor(EnumSet.of(ClientChannelEvent.CLOSED), TimeUnit.SECONDS.toMillis(30));

        System.out.println("Final Out: " + ce.getOut().toString());
    }catch (IOException e) {
        throw new SshClientException("Failed to execute command", e);
    } finally {
        try {
            clientSession.close();
        } catch (IOException e) {
            System.out.println("Got exception while closing session");
            e.printStackTrace();
        }
    }
}

}
Output:
Out1 : org.apache.sshd.common.channel.ChannelPipedOutputStream@26d394cf
Out2 ChannelOutputStream[ChannelExec[id=0, recipient=0]-ClientSessionImpl[root@/:22]] SSH_MSG_CHANNEL_DATA
Out3 org.apache.sshd.common.channel.ChannelPipedOutputStream@26d394cf
Out1 : org.apache.sshd.common.channel.ChannelPipedOutputStream@26d394cf
Out2 ChannelOutputStream[ChannelExec[id=0, recipient=0]-ClientSessionImpl[root@/:22]] SSH_MSG_CHANNEL_DATA
Out3 org.apache.sshd.common.channel.ChannelPipedOutputStream@26d394cf
Final Out: org.apache.sshd.common.channel.ChannelPipedOutputStream@26d394cf
Disconnected from the target VM, address: '127.0.0.1:57678', transport: 'socket'

Process finished with exit code 0

[uttamgupta]

Thanks for helping. I found my mistake and it worked now. Only issue is that sometime outputStream doesn't have data because sever takes sometime to process. If I add sleep for 1 sec and reset stream for every command then I get proper output. Is there any way to find when data is available? I have tried - ce.waitFor(EnumSet.of(ClientChannelEvent.STDOUT_DATA), TimeUnit.SECONDS.toMillis(10)); but it doesn't work, it just returns ClientChannelEvent.OPENED.

Corrected code -
public class InteractiveExecCmd {
public static void main(String[] args) throws Exception {

SshClient client = SshClient.setUpDefaultClient();
client.start();
String hostname = "<hostIp>";
String password = "<password>";
ConnectFuture cf = client.connect("root", hostname, 22);
ClientSession clientSession = cf.verify().getSession();
clientSession.addPasswordIdentity(password);
clientSession.auth().verify(TimeUnit.SECONDS.toMillis(30));
ChannelExec ce;
try {
    ce = clientSession.createExecChannel("bc");

   ByteArrayOutputStream out = new ByteArrayOutputStream();
   ByteArrayOutputStream err = new ByteArrayOutputStream();
   // ByteArrayInputStream inputStream = new ByteArrayInputStream(GenericUtils.EMPTY_BYTE_ARRAY);

   ce.setOut(out);
   ce.setErr(err);
    //ce.setIn(inputStream);
    OpenFuture openFuture = ce.open().verify(30, TimeUnit.SECONDS);
    Stack<String> cmds = new Stack<>();
    cmds.add("4*2");
        cmds.add("7+2");
        cmds.add("7*2");
    List<String> cmdOutputs = new ArrayList<>();
    OutputStream stdin = ce.getInvertedIn();
    Thread t1 = new Thread(() -> {
            // Code to be executed in the new thread
            try {
                while (!cmds.empty()) {
                    String cmd = cmds.pop();
                    stdin.write((cmd + "\n").getBytes());
                    stdin.flush(); // Flush the input to ensure it's sent
                   System.out.println("Sent: " + cmd);
                   Thread.sleep(1000);
                  
                     cmdOutputs.add(ce.getOut().toString());
                    System.out.println("Out1 : " + ce.getOut().toString());
                    out.reset();
                }
            } catch (IOException e) {
                throw new RuntimeException(e);
            } catch (InterruptedException e) {
                throw new RuntimeException(e);
            }
        });
        t1.start();
        t1.join();
   Set<ClientChannelEvent> events =
                ce.waitFor(EnumSet.of(ClientChannelEvent.CLOSED), TimeUnit.SECONDS.toMillis(10));
        System.out.println("Final Output list: " + cmdOutputs);

    } catch (IOException e) {
        throw new SshClientException("Failed to execute command", e);
    } finally {
        try {
            clientSession.close();
        } catch (IOException e) {
            System.out.println("Got exception while closing session");
            e.printStackTrace();
        }
    }
}

}

OutPut:

Sent: 7*23
Out1 : 161

Sent: 723+323
Out1 : 1046

Sent: 4*2
Out1 : 8

Final Output list: [161
, 1046
, 8
]

Process finished with exit code 0

[tomaswolf]

Only issue is that sometime outputStream doesn't have data because sever takes sometime to process.

That's because you use ChannelExec.setOut() and then ByteArrayOutputStream.toString(). Better: don't call ChannelExec.setOut() at all. Instead use ChannelExec.getInvertedOut() and use a BufferedReader around the resturned InputStream and read lines. The read will block until there is data available.

[uttamgupta]

Thanks for helping. Code below is working now. I have few questions though.
1- Do I need to create a thread for executing command? Actually it works without thread also. Actually document says that create a thread. So I wanted to know best practices for this situation.
2- How to get output of first command which runs in ce = clientSession.createExecChannel("bc");
- expected output -
bc 1.07.1

Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006, 2008, 2012-2017 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.

3- How do we know that which output of which command from this block of code -

while ((line = reader.readLine()) != null) {
                    System.out.println("bc output: " + line);
                }

Currently, it is easy to find because each command produces a one-line output, but what will happen when the given command produces multiple lines of output? Sorry if I’m asking any dumb questions, and I appreciate your patience.

public class InteractiveExecCmd {
    public static void main(String[] args) throws Exception {

        SshClient client = SshClient.setUpDefaultClient();
        client.start();
        String hostname = "<hostIP>";
        String password = "<pwd>";
        ConnectFuture cf = client.connect("root", hostname, 22);
        ClientSession clientSession = cf.verify().getSession();
        clientSession.addPasswordIdentity(password);
        clientSession.auth().verify(TimeUnit.SECONDS.toMillis(10));
        ChannelExec ce;

        try {
            ce = clientSession.createExecChannel("bc");
            OpenFuture openFuture = ce.open().verify(30, TimeUnit.SECONDS);
            Stack<String> cmds = new Stack<>();
            cmds.add("`");
            cmds.add("723+323");
            cmds.add("734334*23343434");
            // Create a new thread to handle input
            OutputStream stdin = ce.getInvertedIn();
            Thread t1 = new Thread(() -> {
                // Code to be executed in the new thread
                try {
                    while (!cmds.empty()) {
                        String cmd = cmds.pop();
                        stdin.write((cmd + "\n").getBytes());
                        stdin.flush(); // Flush the input to ensure it's sent
                        System.out.println("Sent: " + cmd);
                    }
                    stdin.write("quit\n".getBytes()); // Exit the bc session when done
                    stdin.flush();
                } catch (IOException e) {
                    throw new RuntimeException(e);
                }
            });
            t1.start();
            // Capture and print the output from bc
            InputStream outputStream = ce.getInvertedOut();
            InputStream errorStream = ce.getInvertedErr();
            BufferedReader reader = new BufferedReader(new InputStreamReader(outputStream));
            String line;
            if (outputStream != null) {
                // Read output from bc

                while ((line = reader.readLine()) != null) {
                    System.out.println("bc output: " + line);
                }
            }
            if (errorStream != null) {
                BufferedReader errorReader = new BufferedReader(new InputStreamReader(errorStream));
                // Read error from bc
                while ((line = errorReader.readLine()) != null) {
                    System.out.println("bc error: " + line);
                }
            }
            Set<ClientChannelEvent> events =
                    ce.waitFor(EnumSet.of(ClientChannelEvent.CLOSED), TimeUnit.SECONDS.toMillis(2));
            t1.join();
        } catch (IOException e) {
            throw new SshClientException("Failed to execute command", e);
        } finally {
            try {
                clientSession.close();
            } catch (IOException e) {
                System.out.println("Got exception while closing session");
                e.printStackTrace();
            }
        }
    }
}

Output:
Sent: 734334*23343434
Sent: 723+323
Sent: bc output: 17141877262956 bc output: 1046 bc error: (standard_in) 3: illegal character:

Process finished with exit code 0

[tomaswolf]

Currently, it is easy to find because each command produces a one-line output, but what will happen when the given command produces multiple lines of output?

This is not a dumb question at all. It shows that the "protocol" for communicating with "bc" is badly defined since "bc" gives no indication where its output for a command ends. If "bc" gave a prompt whenever it expected input, it would be easier. (Such as writing "> " on stdout when it wants new input.) But it doesn't.

I don't have the time to write this code for you. You'll have to figure it out yourself. The main thing is that your code has to keep consuming stdout and stderr, otherwise the command may hang when transport buffers are full. How to synchronize what is read from stdout and stderr with commands you send via stdin is then the application's business. (Same as with any java.lang.Process.) I can't write your application.

As for providing complete examples: we don't have the resources to produce such examples. If someone provides PRs with high-quality examples, we can include them.

As for ChannelEvent.STDOUT_DATA not working: first, I don't think it'd help for this problem (the problem is not when to read since the read will block anyway until data is available but to know when to stop reading, i.e., knowing which was the last line of the output for a particular command), and second, it looks as if this event is never produced in Apache MINA sshd. The latter might be a minor bug in the code; but the usefulness of that event is questionable anyway. The event was apparently introduced very long ago (before 2008), but never implemented.

[uttamgupta]

Thanks for your time and responses. I have one final question: I was just using the command bc as an example, but it could be any command. Another use case could be when we are installing a package on a remote machine, and the installer prompts for a response like 'Yes,' 'No,' or 'All,' or other types of questions and answers. Should I use PTY in these scenarios? If so, how should I configure it?