There is a lot of potential for generating on the fly skins and specs. Everyone seems to want to do stupid flashy junk, but a well configured agentic setup could easily get constrained in ways that alter geometry and interactions more dynamically than the broad scope nonsense I keep seeing.
I haven’t looked into the issue of PCIe lanes and the GPU.
I don’t think it should matter with a smaller PCIe bus, in theory, if I understand correctly (unlikely). The only time a lot of data is transferred is when the model layers are initially loaded. Like with Oobabooga when I load a model, most of the time my desktop RAM monitor widget does not even have the time to refresh and tell me how much memory was used on the CPU side. What is loaded in the GPU is around 90% static. I have a script that monitors this so that I can tune the maximum number of layers. I leave overhead room for the context to build up over time but there are no major changes happening aside from initial loading. One just sets the number of layers to offload on the GPU and loads the model. However many seconds that takes is irrelevant startup delay that only happens once when initiating the server.
So assuming the kernel modules and hardware support the more narrow bandwidth, it should work… I think. There are laptops that have options for an external FireWire GPU too, so I don’t think the PCIe bus is too baked in.
deleted by creator
I prefer to run a 8×7b mixture of experts model because only 2 of the 8 are ever running at the same time. I am running that in 4 bit quantized GGUF and it takes 56 GB total to load. Once loaded it is about like a 13b model for speed but is ~90% of the capabilities of a 70b. The streaming speed is faster than my fastest reading pace.
A 70b model streams at my slowest tenable reading pace.
Both of these options are exponentially more capable than any of the smaller model sizes even if you screw around with training. Unfortunately, this streaming speed is still pretty slow for most advanced agentic stuff. Maybe if I had 24 to 48gb it would be different, I cannot say. If I was building now, I would be looking at what hardware options have the largest L1 cache, the most cores that include the most advanced AVX instructions. Generally, anything with efficiency cores are removing AVX and because the CPU schedulers in kernels are usually unable to handle this asymmetry consumer junk has poor AVX support. It is quite likely that all the problems Intel has had in recent years has been due to how they tried to block consumer stuff from accessing the advanced P-core instructions that were only blocked in microcode. It requires disabling the e-cores or setting up a CPU set isolation in Linux or BSD distros.
You need good Linux support even if you run windows. Most good and advanced stuff with AI will be done with WSL if you haven’t ditched doz for whatever reason. Use https://linux-hardware.org/ to see support for devices.
The reason I mentioned avoid consumer e-cores is because there have been some articles popping up lately about all p-core hardware.
The main constraint for the CPU is the L2 to L1 cache bus width. Researching this deeply may be beneficial.
Splitting the load between multiple GPUs may be an option too. As of a year ago, the cheapest option for a 16 GB GPU in a machine was a second hand 12th gen Intel laptop with a 3080Ti by a considerable margin when all of it is added up. It is noisy, gets hot, and I hate it many times, wishing I had gotten a server like setup for AI, but I have something and that is what matters.
I think authoritarianism is a giant mistake and only creates duplicitous behavior. In my opinion tracking is ridiculous. None of us existed like this and ended up fine. In my opinion, all of this nonsense is acting as a stand in for relationships and real parenting. Humans make decisions and develop ethics based upon trust and autonomy. By stealing that factor of trust and autonomy, and replacing it with authoritarianism a parent is stunting the child’s growth of independent ethics and character. Make compelling discussions of why they should do whatever thing, but let them decide their own path. The lack of compelling discussions and real trust that requires risk is a major factor in the problems that exist in the present world.
The one time you actually need to know where your kid is at because something has happened, you will not know because you have taught them that the only path to independence is to turn off the device and put it into a Faraday cage like pouch, or someone else will do so. If you have a fundamentally trusting relationship with open dialog and respect for their autonomy, they will tell you openly exactly where they are going and any potential for danger. If you can handle that information without allowing anxiety to overwhelm reasoning skills, you will be in a far better position to help them if something bad happens.
The most long term valuable aspect of schooling is the development of one’s social network and connections, along with the habits and ethics. The actual information learned is rather limited in valuable application in the end. Who one knows and how one appears to others is of far more value than what one knows. For these reasons, there may be value in corporate social media. Simply teach the kid to understand how these places are both a trap and a tool. A trap, in that many of the smartest humans are manipulating users in ways that are nearly impossible for the users to escape. Never invest emotions into such a trap. Use the tool if needed for external social benefits, but use it as a manipulation tool with a layer of disconnect from who you really are. Teach them to use a work profile to isolate any apps from their device. That is just how I look at the issue.
It is more complicated than just price. It is ultimately an intuitive self awareness and scope thing. People lack depth to understand the details or ask others that do understand before they make a purchase. The majority of people are more oriented towards interpersonal interactions and experiential aspects of life in their fundamental functional thought. They struggle to see detail and nuances or question fixation and biases.
We still live in the early era of human tribal primitivism when it is quite easy to exploit tribal stupidity on multiple fronts. For some it is fixation from initial exposure or emotional brand perception, others it is impulsive availability, for others they are masochistic misers. Abstractive thinking and understanding is rare in humans, and the majority do not understand it or value it in others.
Walmart bikes are targeting misers first, but spontaneous availability and access, along with controlling the perception of what the low bar of the market is are major factors as well. Each of these three factors exploits a specific niche. Walmart is a rogue wholesale distributor selling directly to consumers using massive capital. They are privateers (legal pirates) in the retail market as are most big box stores. Piracy has always been a nice short term business model for gains. It just happens to be true that people of today like being raided raped and pillaged so long as it is done slowly enough without violence, the ship looks pretty and the pirates wear a suit. Even worse is when pirates become entrenched as monarchs and feudal lords. This is the next step in the evolution when piracy is normalized. Welcome to neo feudalism.
It is simply an entry level thing. You will find this in every market.
In a bike shop retail market I can sell you a serviceable bike for $500 that will last, or an $800 road bike you’ll actually ride. Still the majority of bikes sold come from places like Walmart where they are made of unserviceable junk and are mostly nonfunctional. These are rarely ever ridden and often thrown away. In the shop I’ll sell 20:1 on the cheapest model to the next options up the ladder.
It is strange to adapt to this kind of understanding at first, like just how skewed the real market is. I can target selling to clubs and teams but I can’t touch the the garbage bike market where most people reside.
I think we are at a point where the influx of people into 3d printing are not real Makers or have any aspirations to be.
The reality is that people are often simply stupid. They seem to think that saving a few bucks here or there is smart but are not bright enough to see that everyone doing the same thing are buying the junk product over and over. There is nothing more expensive than being a cheap miser.
Ultimately, the only person that can fix stupid is ourselves. One can only inspire others to learn but can never force them. You cannot fix stupid in others. In the USA, stupidity is political currency and we have a long tradition of poor education and standardized exploitation. It is the American dream.
I think LDO and Voron are the only super relevant open source torchbearers.
In a SBC it is hard to do real-time stuff but there is access to the much faster processor and far more advanced cores and arithmetic logic units. This makes it possible to add more shaping into the input for directions. So each axis can move very quickly near the limits of how fast the physical hardware is capable. The calculations are made to ramp the speed up and down in ways that a little 8/16 bit microcontroller is incapable of achieving. This is also why printers with a 32 bit micro are a little faster as well. The microcontrollers used are like 16-72 MHz but there is no overhead like with an operating system. However, they are also running the PID control algorithms for the bed and hotend. You need both a SBC and a microcontroller unless you get into super niche setups. OS kernel configurations have issues with real-time tasks due to some of the ways kernel space is abstracted in an OS and how the CPU scheduler juggles running process threads and interrupts in the OS and hardware. People do not typically mess with a SBC on this level like adding core isolation with a dedicated thread with the CPU scheduler set to real time. There are other potential factors like core spin up, temperature, and power management that need addressing in the kernel too for RT. This is as far as I understand it, as this is a curiosity I’ve barely scratched the surface of a few times. Hopefully this abstract overview kinda helps.
Think of a microcontroller like a simplified computer from the late 1980s. It is about like an original Nintendo Gameboy but all the extras like memory and RAM are built into a single little chip and the architecture is simplified a little bit. Something like a Rπ SBC is about the same class as a 10yo smartphone. It is actually a TV set top box tuner chip with all the set top box stuff ignored and undocumented.
Marlin is like Arduino firmware. It is just a project that is well organized and setup with an extensive configuration menu about like configuring the Linux kernel. You are prompted with options and you select what is relevant. This is then compiled in a Makefile and you upload the binary to the microcontroller just like an Arduino. The software is setup to make it easy to add similar hardware and maximize entry points so that you can try novel stuff. Unfortunately, Prusa does not run Marlin like this. They are on their own branch of Marlin that specifically makes it difficult to configure and make changes. It also makes cloning a Prusa impossible in practice because they can make changes that will break compatibility. This is the underlying reason the real hobby hacker community that originated around RepRap and the MKx name moved to projects like Voron. The limitations and changes to Marlin were due to Prusa not wanting to break upgrade compatibility and sticking with the AVR microcontroller all the way up to the MK4. They pushed the micro really hard to do both the printer and multi material stuff along with all the fine tuning. So that is kinda the legacy reason for how things evolved.
Personally, I don’t care that my printer is a slow MK3S+. It works well without ever doing a calibration any more and I can print PLA, PETG, TPU, PC, and PA/ABS/ASA with a few caveats. I don’t run my printer 24/7 or even daily, so I am slower than the machine.
I got a little Kingroon KP3S to mess around with Klipper and see if I wanted to build a Voron. I decided not to. Running Klipper means you must setup and dial in all the fine tuning details that Prusa is doing for you with the original firmware. You lose the it just works factor. That is totally fine if your priorities align with this methodology. The KP3S is capable of running Klipper on the original board after just adding the Rπ and loading the firmware. That is probably the cheapest half decent way to mess around with a project printer in Klipper. I never use the thing though.
The Graphene OS homepage has a Matrix chat page configured like this that automatically generates a guest user
(As a mod) Sorry, it must have happened right as I went to bed. So it was the worst timing for a maximum delay. I have removed the duplicates.
I’m not sure why it happened but it probably has something to do with LW being on a very old version of Lemmy and not updating.
ABS needs an enclosure for anything above around 10 layers. Even a room closed with it warm and no one inside is not enough to save an ABS print. Just the air from the moving tool head and the bed are enough to disturb a print and cause layer separation. An IKEA Lack table and a garbage bag over it is enough of an enclosure to count and get most prints alright. It stinks though.
TPU will have holes and look terrible unless you print out of a filament drier. You can dry the stuff a lot and print for around 45 minutes with it in open air before it will absorb enough moisture to start expanding steam in the melt zone and blowing holes in your print layers.
What kind of monster family had a kid with mental health issues, in therapy, and has an accessible gun around unsupervised?
deleted by creator