Go

Go on z/OS

Go on z/OS

This group is a space for open discussions and sharing technical information, and also provides a connection to the wider Go open source community for anyone who is interested in Go on z/OS

 View Only

  • 1.  Codepage conversion issue

    Posted 05/23/24 01:51 AM

    So I've gotten the "micro" editor (https://github.com/zyedidia/micro) running well.  I've only found an issue with one nagging little thing.  There is a function that allows opening of a psuedo-terminal within a micro application window.  In order to make this work I had to make changes to various different programs/APIs.  The issue is that when you write to the tty and then read back from the pty, the original text is in UTF-8 and the text read from the pty is in EBCDIC.  I'm not sure how this may be resolved.  Any thoughts are welcome.

    Here are the changes I made:

    • In project micro:
      • Added zos build tag to internal/action/actions_posix.go and internal/action/terminal_supported.go and !zos tag to internal/action/terminal_unsupported.go.
      • In go.mod, changed version of github.com/mattn/go-isatty required from v0.0.11 to v0.0.13 (which already has z/OS support).
    • In project https://github.com/zyedidia/terminal:
      • Added zos build tags to ioctl_posix.go and vt_posix.go.
    • In https://github.com/zyedidia/tcell:
      • Created tscreen_zos.go from tscreen_solaris.go.
    • In project https://github.com/creack/pty: I
      • Added the !zos build tag to pty_unsupported.go
      • Added new file pty_zos.go, which is as follows:
    //go:build zos
// +build zos

package pty

import (
        "golang.org/x/sys/unix"
        "os"
        "syscall"
)

func open() (pty, tty *os.File, err error) {
        ptmxfd, err := unix.Posix_openpt(unix.O_RDWR)
        if err != nil {
                return nil, nil, err
        }

        p := os.NewFile(uintptr(ptmxfd), "/dev/ptmx")
        if p == nil {
                return nil, nil, err
        }

        // In case of error after this point, make sure we close the ptmx fd.
        defer func() {
                if err != nil {
                        _ = p.Close() // Best effort.
                }
        }()

        sname, err := unix.Ptsname(ptmxfd)
        if err != nil {
                return nil, nil, err
        }

        _, err = unix.Grantpt(ptmxfd)
        if err != nil {
                return nil, nil, err
        }

        if _, err = unix.Unlockpt(ptmxfd); err != nil {
                return nil, nil, err
        }

        ptsfd, err := syscall.Open(sname, os.O_RDWR|syscall.O_NOCTTY, 0)
        if err != nil {
                return nil, nil, err
        }

        t := os.NewFile(uintptr(ptsfd), sname)
        if err != nil {
                return nil, nil, err
        }

        return p, t, nil
}

    The only changes that really matter (as far as I can tell) are the ones to PTY.  If you run "go test" after making the changes above you should see the issue present itself (as well as one other that doesn't seem to matter much.)



    ------------------------------
    Frank Swarbrick
    ------------------------------


  • 2.  RE: Codepage conversion issue

    Posted 05/23/24 09:39 AM
    Edited by Hyder Naqvi 05/23/24 10:22 AM

    Hello Frank that's an interesting problem.
    One solution that comes to mind is to use EBCDIC to ASCII conversion functions we have in ibmruntimes/go-recordio. 
    Here's a codepage conversion example.
    https://github.com/ibmruntimes/go-recordio/blob/main/v2/example-codepage/main.go


    Please let me know your thoughts on using go-recordio as a solution.



    ------------------------------
    Hyder Naqvi
    ------------------------------



  • 3.  RE: Codepage conversion issue

    Posted 05/23/24 10:56 AM

    Yes, of course that is needed.  I'm just not sure where it belongs.  It seems like there should be a general solution, maybe within the Go runtime itself.  I don't know where it would go in the PTY package.  The PTY package includes only the following:

    func Getsize(t *os.File) (rows, cols int, err error)
func InheritSize(pty, tty *os.File) error
func Open() (pty, tty *os.File, err error)
func Setsize(t *os.File, ws *Winsize) error
func Start(cmd *exec.Cmd) (*os.File, error)
func StartWithAttrs(c *exec.Cmd, sz *Winsize, attrs *syscall.SysProcAttr) (*os.File, error)
func StartWithSize(cmd *exec.Cmd, ws *Winsize) (*os.File, error)
func GetsizeFull(t *os.File) (size *Winsize, err error)

    It's not involved with the reads and writes themselves.



    ------------------------------
    Frank Swarbrick
    ------------------------------

    Original Message


  • 4.  RE: Codepage conversion issue

    Posted 05/27/24 11:13 PM

    So I figured out the codepage issue.  I was searching through some of the z/OS Open Tools ports and found this in vimport:

    diff --git a/src/pty.c b/src/pty.c
index f11a22dcd..b8dea655f 100644
--- a/src/pty.c
+++ b/src/pty.c
@@ -194,6 +194,12 @@ mch_openpty(char **ttyn)
     if ((f = posix_openpt(O_RDWR | O_NOCTTY | O_EXTRA)) == -1)
 	return -1;
 
+#ifdef __MVS__
+    // Needed for z/OS so that the characters are not garbled if ptyp* is untagged
+    struct f_cnvrt cvtreq = {SETCVTON, 0, 1047};
+    fcntl(f, F_CONTROL_CVT, &cvtreq);
+#endif
+
     // SIGCHLD set to SIG_DFL for grantpt() because it fork()s and
     // exec()s pt_chmod
     sigcld = mch_signal(SIGCHLD, SIG_DFL);

    I first made some changes to golang.org/x/sys/unix:

    diff --git a/unix/syscall_zos_s390x.go b/unix/syscall_zos_s390x.go
index 312ae6a..84fefae 100644
--- a/unix/syscall_zos_s390x.go
+++ b/unix/syscall_zos_s390x.go
@@ -511,6 +511,7 @@ func Faccessat2(dirfd int, path string, mode uint32, flags int) (err error) {
 //sys  Fchown(fd int, uid int, gid int) (err error)
 //sys  Fchownat(fd int, path string, uid int, gid int, flags int) (err error) = SYS___FCHOWNAT_A
 //sys  FcntlInt(fd uintptr, cmd int, arg int) (retval int, err error) = SYS_FCNTL
+//sys  FcntlFcnvrt(fd uintptr, cmd int, arg *F_cnvrt) (retval int, err error) = SYS_FCNTL
 //sys  Fdatasync(fd int) (err error) = SYS_FDATASYNC
 //sys  fstat(fd int, stat *Stat_LE_t) (err error)
 //sys  fstatat(dirfd int, path string, stat *Stat_LE_t, flags int) (err error) = SYS___FSTATAT_A
@@ -728,6 +729,18 @@ func Ptsname(fd int) (name string, err error) {
        return
 }
--- a/unix/ztypes_zos_s390x.go
+++ b/unix/ztypes_zos_s390x.go
@@ -377,6 +377,12 @@ type Flock_t struct {
        Pid    int32
 }

+type F_cnvrt struct {
+       Cvtcmd int32
+       Pccsid int16
+       Fccsid int16
+}
+
 type Termios struct {
        Cflag uint32
        Iflag uint32

    Then in the new pty_zos.go file in  https://github.com/creack/pty I added the following right after the call to unix.Posix_openpt():

            // Needed for z/OS so that the characters are not garbled if ptyp* is untagged
        cvtreq := unix.F_cnvrt{Cvtcmd: unix.SETCVTON, Pccsid: 0, Fccsid: 1047}
        if _, err = unix.FcntlFcnvrt(uintptr(ptmxfd), unix.F_CONTROL_CVT, &cvtreq); err != nil {
                return nil, nil, err
        }

    There remains one issue, however.  It's treating EBCDIC NL (x15) as just a linefeed.  That is, instead of placing the cursor at the beginning of the next line, it places at the same position as the end of the previous line, just down one line.

    VIM seems to handle this correctly, so I'm sure it's fixable.  Just have to figure out how.

    By the way, is it OK for me to do a PR against golang.org/x/sys?  Or is that something the z/OS Go team would want to do?



    ------------------------------
    Frank Swarbrick
    ------------------------------

    Original Message


  • 5.  RE: Codepage conversion issue

    Posted 05/28/24 05:07 PM

    Frank, you should retrieve the tag on the file descriptor and check its value and text bit and only apply fcntl if it is really needed.



    ------------------------------
    Cw Cheung
    ------------------------------

    Original Message


  • 6.  RE: Codepage conversion issue

    Posted 05/28/24 05:13 PM

    Yeah, I was wondering about that.  Since vimport didn't have it, I chose not to.  But you're probably right.



    ------------------------------
    Frank Swarbrick
    ------------------------------

    Original Message


  • 7.  RE: Codepage conversion issue

    Posted 05/29/24 07:05 AM

    Frank, it is ok for you to do a PR against golang.org/x/sys, the usually practice is that you also add a test case to syscall_zos_test.go for the new functions you added. run `go run mksyscall_zos_s390x.go` on a 3.1 system to recreate zsyscall_zos_s390x.go. 



    ------------------------------
    Cw Cheung
    ------------------------------

    Original Message


  • 8.  RE: Codepage conversion issue

    Posted 06/01/24 03:18 PM

    Instead of creating FcntlFcnvrt, I created Fcntl and third argument is set to interface{} type. I can send you a delta if you want.



    ------------------------------
    Cw Cheung
    ------------------------------

    Original Message


  • 9.  RE: Codepage conversion issue

    Posted 06/01/24 03:32 PM

    Yes please!  (Though I imagine I could do it myself.)



    ------------------------------
    Frank Swarbrick
    ------------------------------

    Original Message


Related Content