Static analysis fixes

[ejohnstown]

Fix some bugs found by the wolfSSL static analyzer:

• Uninitialized mode Variable in FatFS ff_open
• Operator Precedence Bug
• Missing wc_ecc_init() Before ECC Key Import
• Unchecked wc_InitRsaKey Return Value
• Missing break between switch cases
• Missing ForceZero on plaintext password copy

[Copilot]

Pull request overview

Addresses several findings from the wolfSSL static analyzer across terminal handling, SFTP/FatFS file open logic, agent signing key setup, and SSHD authentication cleanup to improve correctness and safety.

Changes:

• Fixes a missing break in control-sequence handling to prevent unintended switch fallthrough.
• Updates SFTP/FatFS open-mode handling and corrects an operator-precedence issue in Win32 authentication.
• Adds missing crypto initialization/checks (ECC init before import, check wc_InitRsaKey() return) and securely zeroes plaintext password buffers before free.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
src/wolfterm.c	Adds missing break to stop unintended fallthrough in CSI J handling.
src/wolfsftp.c	Initializes mode and adjusts flag tests for FatFS open-mode selection.
src/agent.c	Adds unused-parameter suppression in stubs; adds RSA init return check and ECC init before key import.
apps/wolfsshd/auth.c	Zeroes plaintext password copy before freeing; fixes precedence in LsaLookupAuthenticationPackage result check.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

src/agent.c:702

• If wc_InitRsaKey() fails (i.e., ret != 0), the code still falls through to the encSigSz <= 0 check (since encSigSz remains 0) and overwrites ret with WS_CRYPTO_FAILED. This loses the original failure code and can misreport the root cause. Suggestion: handle the wc_InitRsaKey() error via an early return (or preserve ret and skip the encode/signature error mapping when init fails), so init failures aren’t masked as “Bad Encode Sig.”

    int encSigSz = 0;
    int ret = 0;

    ret = wc_InitRsaKey(&key, heap);
    if (ret == 0) {
        mp_read_unsigned_bin(&key.n, rawKey->n, rawKey->nSz);
        mp_read_unsigned_bin(&key.e, rawKey->e, rawKey->eSz);
        mp_read_unsigned_bin(&key.d, rawKey->d, rawKey->dSz);
        mp_read_unsigned_bin(&key.p, rawKey->p, rawKey->pSz);
        mp_read_unsigned_bin(&key.q, rawKey->q, rawKey->qSz);
        mp_read_unsigned_bin(&key.u, rawKey->iqmp, rawKey->iqmpSz);

        encSigSz = wc_EncodeSignature(encSig, digest, digestSz,
                wc_HashGetOID(hashType));
    }
    if (encSigSz <= 0) {
        WLOG(WS_LOG_DEBUG, "Bad Encode Sig");
        ret = WS_CRYPTO_FAILED;
    }

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

[Copilot]

The access-mode decoding is inconsistent with the prior bit-test logic and can produce incorrect behavior. Masking with WOLFSSH_O_RDWR and then comparing to WOLFSSH_O_RDONLY / WOLFSSH_O_WRONLY is only correct if WOLFSSH_O_RDWR is an “access-mode mask” (like POSIX O_ACCMODE); if WOLFSSH_O_RDONLY/_WRONLY/_RDWR are independent bits (as implied by the previous flag & WOLFSSH_O_* checks), the WRONLY/RDONLY branches may never match and the function can incorrectly return -1. Also, create/trunc/append handling now only applies to (WRONLY|CREAT) and no longer applies to RDWR|CREAT paths that previously set create flags. Suggestion: derive an access-mode value using a dedicated mask (e.g., WOLFSSH_O_ACCMODE or an explicit (WOLFSSH_O_RDONLY|WOLFSSH_O_WRONLY|WOLFSSH_O_RDWR) mask) and apply the create/trunc/append logic for both WRONLY and RDWR where appropriate (matching the previous behavior).