Info on 10Gb Ethernet with optical fiber SFP+ transceivers

Discussion in 'Aluma AC Series CMOS' started by Colin Haig, Mar 18, 2024.

  1. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    8,340
    Location:
    Earth
    Fast New Connections:
    New generation CMOS Active Pixel Sensors with 60 (or more) megapixels demand faster technologies to get the data from the camera into the computer.

    10 Gigabit Ethernet (officially IEEE Standard 802.3ae, commonly shortened to 10GbE or 10GigE) is the best technology for distances beyond the speed and distance limitations of USB 3.0.

    10GbE comes in two cabling types - twisted pair copper cabling (10GBASE-T), or optical fiber cabling (10GBASE-SR). Optical fiber can greater distances than copper cabling.

    At the device, such as an SBIG Aluma camera with 10GbE capability or the Personal Computer with a 10Gigabit SFP+ network card or adapter, we use industry standard 10GBASE-SR transceivers (transmitter/receivers) to convert the electrical signals from the device into infra-red light for the fiber. The SFP+ (small form-factor pluggable +) transceiver is about the size of a finger, and slots into a socket in the device.

    The SFP+ socket is designed to hold the transceiver using a latching mechanism - just push the transceiver into the socket all the way until it clicks into place. It has a latching mechanism to stay in place, and can be removed. It also has a latch to retain the cable.

    The SFP+ 10GBASE-SR transceiver connects to 10Gbps Duplex Multi-Mode Fiber Optic Cable. The cable is Aqua (turquoise) in color, and has a LC connector on both ends, with two fibers, one for transmit, one for receive. Sometimes you will see a OM4 50/125um designation on it.

    CAUTION: Don't look into a powered-up optical transceiver or end of the cable, as infrared laser light may be present.

    The fiber optic cable is flexible, but should not be bent tightly. Usually the manufacturer will specify a minimum bend radius - so don't snap it tightly around corners.

    The LC connectors on each end have small round caps. You need to remove the caps before inserting it into the transceiver. The transceiver will usually make a click sound, and there is a latch to retain the cable.

    Troubleshooting problems:
    #1 the SFP+ transceiver isn't pushed in all the way. Push until it's latch clicks.
    #2 you forgot to take the cap out of the fiber side of the transceiver. Usually it is made of black rubber or plastic and needs to be removed before you can insert the cable.
    #3 you forgot to remove the tiny round caps off the end of the LC connectors on the end of the cable.
     
  2. Kenneth Thomas Menzies

    Kenneth Thomas Menzies Cyanogen Customer

    Joined:
    Oct 10, 2014
    Messages:
    29
    Colin:

    Can the AC455 connect directly to the Primaluce Eagle6 Thunderbolt4 port? Which available adaptor is needed if any? Do you think it can also connect to any computer with a Thunderbolt4 port?

    Ken
     
  3. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    8,340
    Location:
    Earth
    Hi Ken,
    You will need a Thunderbolt 10 gigabit ethernet adapter.
    There's a picture of one here, at the top: https://diffractionlimited.com/product/10gb-ethernet-adapters/

    So it goes:
    Camera transceiver - fiber optic cable - transceiver in Thunderbolt adapter - Eagle 6 Thunderbolt 4 port.

    Eagle 6 - I haven't got my hands on the new Eagle 6 yet, as they just announced it.
    @Doug uses an Eagle 5 Pro and this setup, and he is pretty happy with it.

    Details of a specific model are in the AC455 manual.
     
  4. Doug

    Doug Staff Member

    Joined:
    Sep 25, 2014
    Messages:
    10,473
    Yes the AC455 has been working flawlessly with my Eagle via the Thunderbolt adapter.
     
  5. giranda

    giranda Standard User

    Joined:
    Dec 30, 2014
    Messages:
    34
    Location:
    Doylestown, Pennsylvania, USA
    Hi Doug (Hi Colin and Ken, sorry for partially hijacking the thread),

    I have similar setup as Ken describes (AC455 + PrimaLuceLabs Eagle with Thunderbolt ports. In the AC455 manual you provide detailed information on setting up the 10Gb ethernet port for proper data transfer. I am assuming these are not applicable for the thunderbolt ports? Are there any special instructions for using the thunderbolt ports?

    Regards, Vince
     
  6. Doug

    Doug Staff Member

    Joined:
    Sep 25, 2014
    Messages:
    10,473
    Once you install the Thunderbolt adapter, it will appear in the device manager as an Ethernet port. You will still need to configure that port as described in the user manual.
     
  7. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    8,340
    Location:
    Earth
    In addition to Doug's comment:

    If you are using the Sonnet Technologies Solo 3+ Thunderbolt to Ethernet adapter:

    1. Install it in person, not remotely.

    You'll want to have a monitor, keyboard, and mouse attached to the computer with a Thunderbolt port.
    Don't try doing it remotely.
    Windows has some security settings that may block Thunderbolt devices from working.
    When you plug in the Thunderbolt adapter to the computer, you may get a pop-up notification and you may need to allow permission.
    Remote Desktop or other remote tools does not allow for this.

    2. Device Drivers
    The documentation that came with the Sonnet Technologies Solo 3+ Thunderbolt adapter should tell you how to install the Windows Device Drivers.
    If you can't find it, go here:
    https://www.sonnettech.com/support/kb/kb.php?cat=548
    The latest driver they have is version 3.1.7

    3. Network configuration
    As described in the manual.
    If you are using a Solo 3+ Thunderbolt to Ethernet adapter, this is an actual Ethernet network interface you are adding to the PC.
    Just like if you had a Ethernet to USB adapter, or a PCIe card that gets installed inside.
    You will need to check the Network's settings.
    eg Disable all protocols (like IPv6) except for IPv4.
    Set the Advanced settings for RX Buffers = 4096 etc.

    Technical Reference material:
    You hopefully won't need this, but I'm including it just in case...

    Microsoft Kernel DMA Protection - this may prevent the Thunderbolt interface in the computer from working. Turn it OFF.
    Direct Memory Access allows the Thunderbolt network adapter to blast received data directly into the computer's RAM, bypassing the CPU.
    This is essential for speed - we're moving a huge amount of data with these high megapixel cameras.
    https://learn.microsoft.com/en-us/w...ecurity/kernel-dma-protection-for-thunderbolt
    Sometimes there are BIOS settings in the computer related to this.

    Marvell / Aquantia Chipset-based network cards and adapters
    Some motherboards and many network interface cards have a controller chip made by Marvell (formerly Aquantia). This includes the Atlantic family of chips including AQC100, AQC107, and others.
    The Solo unit has an AQC107.
    Other vendors with 10Gigabit Ethernet from Marvell/Aquantia include: Marvell AQtion, QNAP, Dell, HP, Lenovo, Asus, SuperMicro, and others.
    Marvell has publicly available drivers here:
    https://www.marvell.com/support/downloads.html#
    Categories = Marvell Public Drivers
    Platform/OS = Windows
    Part Number = AQC107

    Intel 520/540/710 Network Cards
    For non-Solo network devices, the maker of the card will have drivers and documentation available.
    The Intel 82599 series of chips are used on many cards and some motherboards.
    For example, the Intel® Ethernet Converged Network Adapter X520-DA2 or similar.
    Intel has drivers available on their support site or through their Driver & Support Assistant.

    If you need anything else, let us know.
     
  8. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    8,340
    Location:
    Earth
  9. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    8,340
    Location:
    Earth
    Intel Driver Update:
    Update 30.1 dated 2025-04-29 is now available for some Intel 82599 based 10GB (and faster) Ethernet cards.
    Your card may be manufactured by an OEM using Intel parts - so check the manufacturer's website first.

    The Intel 82599EN drivers:
    https://www.intel.com/content/www/u...10-gigabit-ethernet-controller/downloads.html
    The Intel 82599ES drivers:
    https://www.intel.com/content/www/u...10-gigabit-ethernet-controller/downloads.html
    Both presently point to:
    https://www.intel.com/content/www/u...el-ethernet-adapter-complete-driver-pack.html
    Release notes (check the drop down has 30.1):
    https://www.intel.com/content/www/u...hernet-controller-products-release-notes.html
    Online user guide:
    https://edc.intel.com/content/www/us/en/design/products/ethernet/adapters-and-devices-user-guide/
    Windows installation section of above:
    https://edc.intel.com/content/www/u...d-devices-user-guide/install-windows-drivers/
     
  10. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    8,340
    Location:
    Earth
    Performance Tip #1 - PCIe Slots for certain network cards
    Some Intel 82599 based network cards require a PCIe 3.0 x8 slot, in order to handle the speed of the data coming from the camera.
    Check with the card manufacturer if this is a hard requirement..

    Just because your motherboard has PCIe 4.0 or 3.0 x16 slots, does not mean they will run at x8 lanes.
    Check the manual for the motherboard.
    Sometimes the graphics card slot will run at up to x16, and the slot furthest away will run at only x4. So you may need to move the card.

    Here's how you can tell what link width the card is running at:
    Device Manager - Network Adapters - Intel(R) 82599 10 Gigabit Network Connection
    Properties. Details - Property = PCI current link width
    PCIe_width4.png

    The good news is most cards will operate ok at a lower link width, but performance may be somewhat slower.
     
  11. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    8,340
    Location:
    Earth
    Performance Tip #2 - Network card Advanced Properties

    High data speeds between the camera and the computer may require some "tuning" of the Network Card's settings.
    This tends to be computer-specific, so "your mileage may vary".
    Here are a few suggestions that might be worth trying:

    Write down anything before changing it.
    Rebooting is sometimes needed for a change to take effect.

    Intel Network cards:
    If you have an Intel PCIe network card with driver version 30 or above, you may wish to try these settings.
    If you have a Sonnetech Solo Thunderbolt, QNAP, or Marvell Aquantic based card, see the next section.

    a) Log Link State
    Log Link State Event = Enabled

    The Windows Event Viewer will have an entry when the network link goes up or down.
    Here's an Information message showing the link was established; and above that you can see a warning, also from ixgbn the Intel 10 GB network card, for a disconnect.
    link_state.png
    Frequent state changes often indicate a bad cable or transceiver.
    It's normal for the link to go down when the camera is powered off.

    b) Receive Performance:
    Speed & Duplex = 10 Gbps Full Duplex
    Jumbo Packet = 9014 Bytes
    Receive Buffers = 4096
    Transmit Buffers = 2048

    We use large packets to move the huge amount of camera data.
    We want the link to run as fast as possible.
    The default speed and pack size is too low (around 1514 bytes), and the computer will not talk to the camera.
    More buffers than the default is usually better; 2048-4096 is generally good.

    Receive Side Scaling = Enabled
    Maximum Number of RSS Queues = 8 Queues
    or try 4 Queues.
    If your computer has lots of cores, it doesn't seem to help going beyond 8.

    c) Interrupt Moderation Rate
    Best choice:
    Interrupt Moderation = Disabled
    Interrupt Moderation Rate = Off


    Second best choice:
    Interrupt Moderation = Enabled
    Interrupt Moderation Rate = Low


    Packets will get lost if the card can't interrupt the CPU often enough. This will cause a performance hit.

    d) Other settings
    The following are generally not useful for our purposes, so I recommend Disabling them.
    IPsec Offload = Disabled
    IPv4 Checksum Offload = Disabled
    UDP Checksum Offload (IPv4) = Disabled
    UDP Checksum Offload (IPv6) = Disabled
    TCP Checksum Offload (IPv4) = Disabled
    TCP Checksum Offload (IPv6) = Disabled
    Packet Priority & VLAN = Disabled

    We use UDP to talk to the camera.

    Marvell based cards and Thunderbolt adapters:

    This is a bit preliminary, but I've found this to work well on a test system.
    Speed & Duplex = 10 Gbps Full Duplex
    Jumbo Packet = 9014 Bytes
    Receive Buffers = 4096
    Transmit Buffers = 2048
    Maximum number of RSS Queues = 4 or 8
    Recv Segment Coalescing (IPv4) = Disabled
    Recv Segment Coalescing (IPv6) = Disabled
    TCP/UDP Checksum Offload (IPv4) = Disabled
    Interrupt Moderation = Disabled
    Downshift retries = Disabled


    Enjoy your equipment!
     
  12. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    8,340
    Location:
    Earth
    Performance Tip #3 - Sonnetech Solo Thunderbolt 3 adapter:
    These settings seem to work well on a couple of test systems, Intel i5 based NUC and PrimaLuce Eagle 4 Pro with the Solo 3+ adapter, using driver version 3.1.10.0 from Marvell public drivers.
    Launch the Device Manager. Expand Network Adapters - Solo 10G SFP+ Thunderbolt 3 Edition
    Right click for Properties. Like so:
    solodm1.png

    On the Advanced Properties tab:
    solo_dm_advanced.png
    ARP Offload = Disabled
    Downshift Retries = Disabled
    Energy Efficient Ethernet = Disabled
    Flow Control = RX & TX Enabled
    Interrupt Moderation = Disabled
    Interrupt Moderation Rate = Disabled
    IPv4 Checksum Offload = Disabled
    Jumbo Packet = 9014 Bytes
    Large Send Offload V1 (IPv4) = Disabled
    Large Send Offload V2 (IPv4) = Disabled
    Large Send Offload V2 (IPv6) = Disabled
    Locally Administered Address = Not Present
    Log Link State Event = Enabled
    Maximum number of RSS Queues = 4 Queues
    NS Offload = Disabled
    Priority & VLAN = Priority & VLAN Disabled
    Receive Buffers = 4096
    Receive Side Scaling = Enabled
    Receive Segment Coalescing (IPv4) = Disabled
    Receive Segment Coalescing (IPv6) = Disabled
    Speed & Duplex = 10 Gbps Full Duplex
    TCP/UDP Checksum Offload (IPv4) = Disabled
    TCP/UDP Checksum Offload (IPv6) = Disabled
    Transmit Buffers = 2048
    VLAN ID = 0
    Wake from power off state = Disabled
    Wake on Link = Disabled
    Wake on Magic Packet = Disabled
    Wake on Pattern Match = Disabled
    Wake on Ping = Disabled

     
  13. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    8,340
    Location:
    Earth
    Performance Tip #4 - Change the Interface Metric
    The "Interface Metric" is used by the operating system to figure out which network interface is preferred.
    For example Windows always tries to switch over to use the fastest network.
    Normally that's fine - eg a wired 1GB connection will be faster than most Wi-Fi connections.

    When we add the 10Gbps Ethernet adapter to your computer to talk to the camera, the operating system will start sending all network traffic over this fast interface first. Some applications may need to be restarted, and overall performance may suffer.


    Solution: Manually change the Interface Metric on the 10Gbps Ethernet connection to a high value, like 1000, so that only camera traffic goes over the fibre connection.

    1. Launch the Control panel: Start... run... cpanel.exe
    Next, you will need to change the adapter settings:
    cpanel_network.png
    2. Click on Network and Internet
    3. Click on Network and Sharing Center
    4. Click on Change adapter settings

    Next:
    cpanel_net_prop.png
    5. Click to select the 10Gbps adapter
    6. Right-click the Properties
    Next:
    cpanel_78.png
    7. Click on Internet Protocol Version 4
    8. Click Properties
    Then:
    cpanel_9.png
    9. Click [Advanced...]
    10. Disable / Turn off (uncheck) [] Automatic Metric
    11. Set the Interface Metric to 1000
    12, 13, 14. Click OK to close each screen.

    Reference: https://learn.microsoft.com/en-us/t...r/networking/automatic-metric-for-ipv4-routes
     

Share This Page