Source   Edit  

os and i/o

bookmark

TLDR

• for async i/o see asyncPar.nim
  • you generally want threadpool imported to getaround the blocking nature of i/o logic
  • this includes executing external processes
• if a proc accepts a filename (string), it may also accept a File/Filehandle
• tips
  • always use absolutePath when working with paths
• generally you should check when defined(posix/linux/etc) and use unixToNativePath
  • extremely relevant if your app supports disparate systems
  • posix
    • admin: root
    • cachedir: XDG_CACHE_HOME | HOME / .cache / app (etc for other dir types)
    • dest paths inherit user default perms
    • dir symlinks copied as symlinks to dest
    • executables not in path may require "executable.sh".normalizeExe to get ./executable.sh
    • file creation time may actually be last modified time
    • file symlinks are followed (by default) then copied to dest
    • osLastError works like $?
    • paramCount not defined when generating dynamic libraries (See --app:lib)
    • parseCmdLine splits on whitespace outside of quotes (use parseopt module instead)
    • path considered hidden based solely on the path string
    • paths are case sensitive
    • permissions are copied after file/dir is -> could lead to race conditions
    • permissions can be set without following symlinks if lchmod is avail and doesnt err
    • relativePath works as expected
    • removeFile errors if readonly
    • tempdir: TMPDIR | TEMP | TMP | TEMPDIR
  • windows
    • admin: admin local group
    • cachedir: LOCALAPPDATA / app / cache (etc for other dir types)
    • dest paths inherit source paths attributes
    • dir & file symlinks are skipped
    • network paths are considered absolute
    • osLastError works like windows i guess
    • parseCmdLine is overly complex (use parseopt module instead)
    • path considered hidden if file exists and hidden attribute set
    • paths are case insensitive
    • relativePath requires startpath & basepath args with same roots
    • removeFile errors ignores read-only attribute
    • require evelated privs for sym/hardlinks
    • tempdir: calls windows GetTempPath
    • permissions set on files have to follow symlinks
  • OSX
    • i think OSX should align with posix sans whatever follows
    • cachedir: XDG_CACHE_HOME | HOME / .cache /app (etc for other dir types)
• osproc
  • calling close before a process has finished may result in zombies and pty leaks
  • generally
    • process streams/filehandles shouldnt be closed directly, but the process itself
    • cmd accepting ENV arg uses the parent process by default
    • cmd accepting workingDir arg uses the current dir by default
  • poUsePath > poEvalCommand for portability and let nim escape cmd args correctly
  • refrain from using waitForExit for processes w/out poParemtStreams for fear of deadlocks
• user input
  • commandline params are passed when your app are started
  • use the standard input stream to accept user input thereafter

links

• other
  • nitch source code
  • peter: handling files in nim
• high impact
  • basic os utils
  • distro detection & os pkg manager
  • env support
  • file and string streams
  • fusion file permissions
  • fusion io utils
  • fusion scripting
  • get cpu/cors info
  • i/o multiplexing
  • mem files
  • parse cmdline opts
  • posix wrapper
  • process exec & comms
  • read stdin
  • system io
  • terminal
  • temp files and directories
• niche
  • open users browser
  • raw posix interfacehttps://nim-lang.org/docs/posix.html

TODOs

• cpuEndian
• cpuRelax
• DynlibFormat, ExeExts, ScriptExt
• find instantiationInfo in the docs

system

vars/procs/etc

• hostCPU
  • "i386", "alpha", "powerpc", "powerpc64",
  • "powerpc64el", "sparc", "amd64", "mips",
  • "mipsel", "arm", "arm64", "mips64", "mips64el", "riscv32", "riscv64"
• hostOS
  • "windows", "macosx", "linux", "netbsd",
  • "freebsd", "openbsd", "solaris", "aix", "haiku", "standalone"
• FileMode = enum
  • fmRead only
  • fmWrite zero file (force creates) then open for writing
  • fmReadWrite zero file (force creates) then open for rw
  • fmReadWriteExisting same but doesnt create file
  • fmAppend append doesnt create file
• getFreeMem number of bytes owned by the process, but do not hold any meaningful data

os

os exceptions

• OSError e.g. file not found, incorrect perms

os types

• CopyFlagenum symlink handling
• DeviceIdint32
• FileIdint64
• FileInfoobject associated with a file object
• FilePermissionenum modeled after nix
• OSErrorCodeint32
• PathComponentenum type of path(file, symlink to (dir/file)) to a file object
• ReadDirEffectobject reading a dir
• ReadEnvEffectobject reading from ram
• WriteDirEffectobject write to a dir
• WriteEnvEffectobject write to ram

os consts

• AltSep '/'
• CurDir '.'
• DirSep '/'
• doslikeFileSystem true
• ExtSep '.'
• FileSystemCaseSensitive false
• invalidFilenameChars {'bunch of stuff'}
• invalidFilenames "bunch of stuff"
• ParDir ".."
• PathSep ';'

os procs

• copyFile dest parentDir must exist; overwrites but preserves perms
• execShellCmd blocks until finished
• createDir mkdir -p
• existsOrCreateDir mkdir
• moveDir doesnt follow symlinks, thus symlinks are moved and not their target
• moveFile (see moveDir) + can be used to rename files
• normalizePath (no d) modifies string inplace
• normalizeExe prefixes path with ./
• osErrorMsg converts an OSErrorCode to a human readable string
• osLastError $?
• newOsError errorCode determines the msg as retrieved by osErrorMsg, get code via osLastError
• raiseOsError

osproc

• advanced cmd execution & (background) process communication

osproc types

• Process ref of an os proces
• ProcessOption enum that can be passed to startProcess
  • poEchoCmd before executing
  • poUsePath to find cmd like this "cmd", args="as", "array"
  • poEvalCommand without quoting using system shell like this "cmd args inline"
  • poStdErrToStdOut 2>&1
  • poParentStreams use parent stream
  • poInteractive optimize buffer handling for UI applications
  • poDaemon execute cmd in background
• poDemon: the best podcast on chartable

osproc procs

• close forcibly terminates the process and cleanup related handles
• countProcessors cpuinfo.countProcessors
• errorHandle of a process for reading
• errorStream of a process for reading; doesnt support peak/write/setOption
• peekableErrorStream of a process
• execCmd returns $?; std0,1,2 inherited from parent
• execCmdEx returns (output, $?); blocks if input.len > OS max pipe buffer size
  • particularly useful as you can set ENV, working dir, and data via stdin in one go
• execProcess use PoUsePath for args syntax
• execProcesses in parallel
• hasData checks process
• inputHandle of a process
• inputStream of a process
• kill a process via SIGKILL on posix, terminate() on windows
• outputHandle of a process; doesnt support peek/write/setOption
• peekableErrorStream of a process
• peekableOutputStream of a process
• peekExitCode -1 if still running, else the actual exit code
• processID of a process
• readLines of bg process invoked with startProcess
• resume a process
• running true if process is running
• startProcess in background
• suspend a process
• stop a process on posix via SIGTERM, windows via TerminateProcess()
• waitForExit of process and return $?

osproc iterators

• lines of process invoked with startProcess

parseopt

• parsing cmd line args

parseopt syntax

• short opt: single dash e.g. -a -a:1 -a=1
  • -a:1 = (kind: cmdShortOption, key: a, val: 1)
  • by default -abc = 3 short options each having value ""
    • providing shortNoVal to initOptParser with any value changes this behavior
    • -abc becomes a=bc
• long opt: double dash e.g. --a --a:1 --a
  • --a:1 = (kind: cmdLongOption, key: a, val: 1)
  • by default --a b c becomes 1 option and 2 args
    • providing longNoVal initOptParser with any value changes this behavior
    • --a b c becomes --a=b arg c
• cmd args: anything not prefixed with - or following --s
  • e.g. -- 1 2 3 results in 3 cmd arguments
  • -- a = (kind: cmdArgument, key: a)
• values: anything after the first : or = e.g. -a:1 -a::1 -a=:1 equals 1 and :1, respectively

parseopt types

• CmdLineKind enum
  • cmdEnd nothing else exists
  • cmdShortOption -blah
  • cmdLongOption --blah
  • cmdArgument blah or -- blah
• OptParser object to collect short, long options and cmd arguments
  • pos int
  • inShortState bool
  • allowWhitespaceAfterColon bool
  • shortNoVal setchar short options with optional values
  • longNoVal seqstring long options with optional values
  • cmds seqstring whitespace delimated strings
  • idx int
  • kind CmdLineKind
  • (key, val) of short, long or cmds depending on kind

parseopt exceptions

• ValueError when initOptParser cant parse cmdline opts

parseopt procs

• initOptParser with cmdline options
• next token is parsed into an OptParser instance
• cmdLineRest of the cmd line that has not been parsed
• remainingArgs that have not been parsed

parseopt iterators

• getOpt from cmdline and return instanceof OptParser